protobuf breaking changes between 3.20.x and 4.21 (4.x)

[SomberNight]

Google made breaking changes in protobuf (see e.g. here), in the generation and parsing of _pb2.py generated files.

Re the python protobuf package, the 3.20.1 release is followed by the 4.21.1 release.

AFAICT protoc >=3.19 generates _pb2.py files in the new format, while older protoc generates the old format. AFAICT

• protobuf 3.20.x is able to parse both the old and the new format,
• protobuf<3.20 can only parse the old format,
• protobuf>=3.21 (i.e. protobuf>=4) can only parse the new format

We rely on protobuf to parse _pb2.py files at runtime:

• directly for paymentrequest_pb2.py
• indirectly via bitbox02 (hww plugin)
• indirectly via keepkey (hww plugin)

---

• Currently, our _pb2.py file(s) is in the old format, and we require protobuf>=3.12,<4.
  • see https://github.com/spesmilo/electrum/issues/7833
  • and https://github.com/spesmilo/electrum/commit/52b73880f95be5cf51742fec10cde0a2e4b23de4
• bitbox02 has _pb2.py file(s)
  • for bitbox02<6.1.0, in the old format
  • for bitbox02==6.1.0, in the new format
    • and it requires "protobuf>=3.21" (i.e. "protobuf>=4")
      • related PR
      • ~~would be nice if they relaxed this to accept 3.20.x~~ (done in PR, thanks!)
  • for bitbox02>=6.1.1, in the new format
    • and it requires "protobuf>=3.20"
• keepkey has _pb2.py file(s) in the old format
  • I've asked them to regenerate in new format and publish a new release
    • see issue

[rt121212121]

I emailed KeepKey a year or so ago and asked if they could update the PyPI package. Their reply was that they had nothing to do with the Python package and to use Github 🤷‍♂️

[SomberNight]

If they don't have the keys to the PyPI package due to changing maintainers or some other reason, they should release a new package under a different name. Alternatively we might accept a PR that makes python-keepkey a git submodule in this repo. Otherwise I am considering removing the plugin...

[SomberNight]

Switched to new format for our own paymentrequest_pb2.py in https://github.com/spesmilo/electrum/commit/4f9469b7894d0e237beaabd62a2b8a6153f6dcdb.

Something needs to be done with keepkey so that we can drop the protobuf<4 upper bound pin.