protobuf
protobuf copied to clipboard
Add standalone protoc plugin binaries
This PR adds targets to build protoc
plugins for all the built-in plugins. This will result in the following binaries being available:
-
protoc-gen-cpp
-
protoc-gen-csharp
-
protoc-gen-java
-
protoc-gen-kotlin
-
protoc-gen-objectivec
-
protoc-gen-php
-
protoc-gen-python
-
protoc-gen-pyi
-
protoc-gen-ruby
-
protoc-gen-rust
The goal of this PR is for these to be added as assets to each GitHub release, although we are unable to find the code that builds the assets for releases - if it is in this repository, let us know and we're happy to do any work required!
There's multiple motivations for this PR:
- Optimally, people want to version their plugins separately from their compiler. That is, the version of
protoc
should be decoupled from the version of each plugin, as is the case with the rest of the Protobuf ecosystem outside of the built-in plugins. You should be able to, for example, take advantage of fixes inprotoc
v24.0 whilst continuing to use the Java generated code from v23.0, in case there are breaking changes you're unable to adopt yet. - This partially addresses https://github.com/protocolbuffers/protobuf/issues/5587 - the core motivation behind this issue is that when piping a
FileDescriptorSet
intoprotoc
with--descriptor_set_in
, and then invoking one of the built-in plugins, different code can result, as the value forjson_name
will be subtly different when taking an externalFileDescriptorSet
vsprotoc
producing one internally and then invoking one of the built-in plugins. - Selfishly, it makes life a lot easier for alternative Protobuf compilers. For example, for us with
buf
, to use the built-in plugins, we have a convoluted mechanism to proxy to protoc that cannot deal with issue #5587, and requires people to install bothprotoc
andbuf
when they may just want one of the built-in plugins. Having separate, independently-versionable plugins would make our lives a lot easier.
The majority of this PR is something we have already been maintaining for our remote plugins feature within Buf, we're just hoping we can get this upstreamed, as we think it would benefit the Protobuf community.
Follow-ups that would be nice:
- Adding to GitHub releases, as mentioned.
- Get these plugins installed as part of the Homebrew
protobuf
formula. There's precedent for this with i.e. thegrpc
formula, which installs each language-specific plugin. - Change
protoc
to default to using an on-disk plugin vs a built-in plugin, that is ifprotoc-gen-cpp
exists, useprotoc-gen-cpp
over the builtincpp
plugin. This allows people to independently version plugins as mentioned, and mirrors the behavior of the Well-Known Types (if a WKT is in the include path, it is used instead of the WKTs included as part of theprotoc
distribution). We're happy to put up a PR for this as well.