buf + dependabot Integration

[hasit]

It would be good to have buf + dependabot integration so that protobuf dependencies managed by buf can be updated at a regular period.

Something like this:

# /path/to/.github/dependabot.yaml
  # Buf
  - package-ecosystem: "buf"
    directory: "/path/to/api"
    schedule:
      interval: "monthly"
    rebase-strategy: "disabled"
    open-pull-requests-limit: 20

I'd be happy to help further on this feature request.

Ref: dependabot config

cc: @robbertvanginkel

[lrewega]

It seems like dependabot is not currently accepting new ecosystems.

That being said, they also mention:

If you are an ecosystem maintainer and are interested in integrating with Dependabot, and are willing to help provide the expertise necessary to build and support it, please open an issue and let us know.

So it looks like the next step would be to file an issue on dependabot/dependabot-core and for Buf to decide if this is something we want to build and support.

[hasit]

@lrewega

Sounds good. If there is further interest in this, I'd be happy to discuss my usecase and help take this forward.

[maxbrunet]

There is also Renovate which is an excellent alternative to Dependabot and is always accepting new managers: https://github.com/renovatebot/renovate/issues/new

Once the feature request is created, they will ask to complete this questionnaire: https://github.com/renovatebot/renovate/blob/main/docs/development/new-package-manager-template.md

On top of schema repositories, support for updating Remote Generation plugins would nice, but it is alpha (would it be too unstable for such integration?): https://docs.buf.build/bsr/remote-generation/overview

If they accept the feature request, contributing is easy, the project is well structured and the maintainers provide helpful guidance.

I am very new to Buf (I am not even sure I can call myself a user, as I am only helping on a project using it), so I do not feel familiar enough to fill out the FR myself yet. But once it is done and if the design is clear and without major complexities, maybe I would have time to help with the implementation.

At first glance, multiple components would need to be written:

• Datasources and managers for BSR repositories, plugins, templates (depending on the Buf architecture and the scope of the FR, it seems to range from 1 datasource / 1 manager to 3 datasources / 3 managers)
• Versioning logic? Plugin versions can have a build number (e.g. -1), so it is not exactly SemVer, although it is not an uncommon format, not sure if an existing one can be reused or a new one is required. (Some repos like googleapis/googleapis use Git commits as tags which could be very inconvenient to compare versions, if that is not the main usage, we can probably ignore this case for now)

[bufdev]

Closing this for now - dependabot has been closed to new ecosystems for over two years now, and doesn't look like it is opening up anytime soon: https://github.com/dependabot/dependabot-core/blob/main/CONTRIBUTING.md. We'd love to add buf to dependabot if we could, so we can revisit this in the future.

[dansheerhealth]

@bufdev Just wanted to flag that they're open to new ecosystems: https://github.com/dependabot/dependabot-core/commit/0f931f452e9e51a3996a010ca21c90115b7f80cf

[bufdev]

Reopening - we'll take a look!

[chrispine]

Thanks for raising this topic again! We spent some time looking into this, and the TL;DR is that we couldn’t find a way to implement this that isn’t a pain for customers to set up, and that provides much value. This is because Dependabot doesn't permit running a custom post script.

One avenue we explored (the one recommended by GitHub) was to create a GitHub action that runs on Dependabot PRs, but this would have to be configured on each repo, and then you’d have to wait for the follow-up commit to review the PR. It’s our guess that most people just won’t want to bother with this amount of hassle for the benefit this provides.

We explored some other options as well, but they were unfortunately not feasible for one reason or another. So at the moment, we’re going to focus on things that will provide greater benefit to a greater number of our users. Thanks for understanding!

[nicksnyder]

Just wanted to add that an alternative solution is to use Generated SDKs. These look like normal packages in your language's ecosystem and will get updated in the same way as your other language dependencies (aka Dependabot works with them if you already have that configured).