ibc-rs icon indicating copy to clipboard operation
ibc-rs copied to clipboard

Migrate the codebase from tendermint-rs dependency to cometbft-rs

Open adizere opened this issue 2 years ago • 2 comments

Feature Summary

We need to test, validate, and ship the upgrade to CometBFT-rs so that we can safely sunset Tendermint-rs eventually.

Tasks

  • [ ] This work is paired with Hermes-side migration
    • https://github.com/informalsystems/hermes/issues/3820
    • the migration on the two repos should proceed in tandem for end-to-end testing that IBC relaying is not impacted by this migration
  • [ ] Announce this testing and the planned migration to all users of IBC-rs (maybe in community call) to give them an early notification

adizere avatar Feb 02 '24 10:02 adizere

The tendermint packages are used as a dependency throughout the cosmos-sdk and interchain-security proto files. The generated files in cometbft-proto correspond to the cometbft.* proto packages as published buf.build/cometbft/cometbft. So the conversion must start with the proto files in the upstream repositories.

mzabaluev avatar Feb 06 '24 12:02 mzabaluev

So the conversion must start with the proto files in the upstream repositories.

There could be a way forward that does not need waiting for upstream protos to convert from the bundled tendermint.* proto files. We can remap the tendermint.* proto packages to the cometbft_proto modules corresponding to the protobuf revisions in the versioned cometbft.* layout, using the extern_path option of prost-build in the proto-compiler.

Unfortunately, as I have just discovered, the upstream proto revisions as currently pulled to build ibc-proto-rs are not consistent as per the revisions of tendermint they expect to import:

  • https://github.com/cosmos/cosmos-sdk/commit/2e9e5d6eea24d6c11eddc9c002c66e89ae036187 uses v0.37
  • https://github.com/cosmos/ibc-go/commit/57fcdb9a9a9db9b206f7df2f955866dc4e10fef4 uses v0.38
  • https://github.com/cosmos/interchain-security/commit/ea545b490b53b8e6f422492b4fb2820ab0bc140b uses v0.37

However, by the way the build process is set up in sync-protobuf.sh, all of these inputs to proto-compiler actually end up including the tendermint/* protobuf files found in the cosmos-sdk buf export (the proto-include directory). Which means our ibc:: generated types for this set of upstream revisions do not necessarily interoperate with the Go types generated in ibc-go as per the targeted commit.

The proper way out of this mess is to convert the upstreams to produce buf modules and reuse buf.build/cometbft/cometbft as a common dependency. Then the build process in ibc-proto-rs could consume the buf modules to generate Rust bindings for them.

Meanwhile, it's possible to juggle the src/*_COMMIT files in ibc-proto-rs to arrive at a combination of proto upstreams that happens to use the same revision of the tendermint protos, and then do the extern_path trick on that. But it's a high-quality balancing act that needs developer's time and care. Which is precisely what buf is meant to get rid of.

mzabaluev avatar Feb 08 '24 18:02 mzabaluev