[RFC 0172] Mergebot

[Lassulus]

Rendered

[0x4A6F]

I'd like to nominate for shepard team.

[philiptaron]

Rendered

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgs-merge-bot-testing-and-plan/39824/13

[FRidh]

Previous proposal on this topic https://github.com/NixOS/rfcs/pull/50.

[Lassulus]

Previous proposal on this topic #50.

ah thanks, https://github.com/NixOS/rfcs/pull/128 also seems to be related, I added them to prior art in the RFC text

[Mic92]

I'd like to nominate for shepard team.

[Aleksanaa]

There is also a less common situation where automatic merge may be possible: maintainers updating their own information in maintainer-list.nix.

[Mindavi]

There is also a less common situation where automatic merge may be possible: maintainers updating their own information in maintainer-list.nix.

I'd say that it's fine to leave that out of scope for now.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-04-02/42643/1

[JulienMalka]

I'd like to self-nominate to be a shepherd.

[infinisil]

I'm also nominating myself as a shepherd

[Lassulus]

alrighty, I'm not sure about the exact process. But we have @0x4A6F @JulienMalka @Mic92 and @infinisil who nominated as shepherd, @Mic92 told me he would be ok with lead. So I guess the next step is to find a suitable date for our first call. If anybody else also wants to join up, please mention it here :)

[infinisil]

While we have enough shepherds, until the RFC steering committee officially announces the shepherds, nominations are still open. But that's not a blocker for setting up a call already :)

[Lassulus]

https://meet.dgnum.eu/rfc172-403705 time to find a date :)

[Mic92]

Ok. It looks like the 16:00 UTC, 19.04.2024 is the best date for us for a first meeting.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-04-16/43512/1

[Mic92]

RFC meeting summary on the 23.04.2024

Attendees: @Mic92 @Lassulus @Scriptkiddi @a-kenji @0xA46F @infinisil

Key Topics Discussed:

1. Trust and Verification for Maintainers:

   • The need to trust the upstream author, a Nixpkgs committer, and with the new RFC, maintainers were emphasized.
   • We discussed various ways to vet maintainers, including establishing a trusted maintainer group with the authority to merge their own package PRs, similar to existing practices but scaled for efficiency.

2. Proposal Enhancements:

   • The idea of a JSON interface to limit maintainers to specific actions was considered to prevent them from altering the source code of packages.
   • Discussions about establishing a category system for changes in a PR were also highlighted, aiming for different vetting requirements based on the type of change, like version bumps.

3. Automation and Efficiency:

   • Several suggestions were made to automate and streamline processes, such as integrating third-party CI results for version bumps automatically and enhancing ofborg for better CI performance.
   • The potential for running NixOS tests more easily and enabling VNC sessions for immediate checks on graphical packages was discussed.

4. Review Processes:

   • The concept of a merge queue managed by committers, as opposed to direct pushes by mergebots, was debated.
   • We explored implementing multi-stage review processes, where maintainers first ensure PRs resolve the issues, followed by a security review by committers.

5. Security Concerns and Workflow Enhancements:

   • The potential risk of maintainers switching sources under the guise of maintaining a fork was acknowledged as a significant security concern.
   • Improvements in labeling and PR checklists were suggested to clarify what aspects of a PR were reviewed and by whom.

Decisions and Actions:

• The group agreed to continue discussions on refining the maintainer vetting process and enhancing automation to reduce manual oversight.
• The next meeting is scheduled for two weeks from now, maintaining the fortnightly meeting rhythm to further these discussions and solidify plans.

Full meeting notes

# RFC172 meeting notes

2024-04-19 16:00 UTC

Attendees: @Mic92 @lassulus @ScriptKiddi @a-kenji @0xA46F @infinisil

next meeting friday in 2 weeks, meetings are aimed fortnightly

• Current people we need to trust:

  • The upstream author
  • A Nixpkgs committer
    • Comes from https://github.com/NixOS/nixpkgs/issues/50105

• With this RFC, maintainers need to be trusted

  • How to make sure that maintainers are vetted
    • How to validate maintainers at all?
    • And how to make sure the committers know about that and enforce it?

• What if there's something like a trusted maintainer group that can merge their own packages PRs

  • Kind of similar how it works today, just more scalable

• Don't allow maintainers to change the source, only the code

• What if there's a JSON interface that restricts what can be done, e.g.

  {
    "builder": "mkDerivation"
  }
  

• As a user, when I install hello, I trust the upstream author of the hello package

• Possible attack: A maintainer could switch the underlying source, arguing that it's the newly maintained fork.

• No self merges?

  • Sock puppet accounts defeat that, can't be enforced
  • Require people to maintain some packages?
    • Can easily be defeated

• Trusted maintainers vetting process

• Establish some category system for the changes done in a PR, different requirements for each category

  • E.g. version bump

• More underlying problem to address? Too much overhead, too many committers losing interest, etc.?

• How to resolve?

  • Single PR that version bumps every packages?
    • Hard to review, changes how packages get reviewed
  • Making it more automated to get feedback to version bumps?
    • E.g. automatically get third-party CI results for version bumps
    • Lassulus: Would still want to check it myself
  • Make it easier to run more involved tests like NixOS tests. Currently not easily doable, very resource intensive
    • There's new recent container testing stuff, neat!
    • But there's software really hard to test
  • Making it easy for people to get into a VNC (remote) session to immediately check out graphical packages
  • Improving ofborg, general PR CI story
    • Building all dependent packages in ofborg
    • Improve evaluation speed, especially to figure out which packages changed

• General theme: Make automation better

• @Fritz: Even if we make automation better, most people just make a couple PRs and don't spend much time after that. Even if we make it 50% more efficient, still not enough time.

  • @Mic92: What if mergebot doesn't push directly, but pushes to a merge queue branch
  • Committers review the queue
  • @infinisil: Doesn't seem realistic
  • @Mic92: Would be a single page instead of many to review
  • @infinisil: I feel like it would be the same work

• @Mic92: As a committer I don't want to have to check whether the PR solves the issue, this should be for maintainers to do

  • @infinisil: What if we have stages of review

    • Zeroth stage: CI checks
      • Commit history
      • Formatting
      • ...
    • First stage: Review by maintainers to ensure it fixes the issue
    • Second stage: Security review
      • For a committer, only security is checked

  • @Mic92: When I get a checkmark from a reviewer, I'm not sure what was reviewed

    • Reviewers should always have to indicate what they did to check the PR

  • @kenji: Like PR checklist but the other way around: Have other people say what they did of the checklist

  • Something with labels, currently everybody can label

    • Implementation not important, can be implemented for sure

Domen (1)

• Committers (200)

  • Trusted maintainers with a vetting process similar to committers (1000?)
  • Maintainers (20000)

• Interesting idea: Reuse Nix files from upstream -> doesn't require trusting anybody else to update Nix files

  • nixpkgs.toml defines package.nix to use, which gets called with callPackage
  • Lots of ideas

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-05-14/45414/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-05-28/46113/1

[nyabinary]

Is a review bot in scope for this as well?

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-06-10/46817/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-06-24/47589/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-07-08/48678/1

[Bot-wxt1221]

As far as I know, the hardest package to review is the desktop application. Unluckily, it is the most important problem that the RFC want to solve. Because they updates too quickly and makes nixpkgs too many PRs. However, I do not think we should provide VNC to maintainers because the desktop applcation is always with many features like audio and so on. The easy VNC can't solve this problem.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-08-05/50170/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfcsc-meeting-2024-08-19/50831/1

[Lassulus]

drafting this until we have time gain (post NCA), in september or october :)