js-waku icon indicating copy to clipboard operation
js-waku copied to clipboard

feat: Deprecate Named Sharding and Update Lightpush Client API

Open chair28980 opened this issue 9 months ago • 11 comments

In the context of discussions around named sharding and peer management for the Waku network, it is proposed to make the following changes to the Waku Network:

  1. Deprecate Support for Named Sharding:
  • It is recommended to remove support for named sharding.
  • Peer management should ensure that only peers supporting sharded topics are connected.
  1. Lightpush Protocol Updates:
  • The lightpush protocol itself should never be aware of the sharded format of pubsub topics. It is an extension of libp2p's gossipsub.
  • On a protocol level, the lightpush service node needs to handle requests on unsupported pubsub topics by returning a clear error code.
  1. Lightpush Client API Changes:
  • To maintain adherence to the sharded format, the lightpush client API should be updated.
  • New proposed format: lightPublish(WakuMessage, ClusterID, ShardNumber)
  • For legacy requests on named topics, the API could use a trial and error process to send the request to random service nodes. This can be done using the format: lightPublish(WakuMessage, PubsubTopic)
  • A request format that bypasses our peer management/selection: lightPublish(WakuMessage, PubsubTopic, ServiceNodeMultiaddr)

  • Users of custom pubsub topics should select a cluster id.
  • Use static sharding until the launch of The Waku Network.
  • Default Pubsub topic support remains for now, to be phased out as a later step

Acceptance Criteria:

  • [x] Remove support for named sharding (except for default pubsub topic)
  • [ ] Update lightpush protocol to handle unsupported pubsub topics with clear error codes.
  • [ ] Introduce the new lightpush client API as mentioned above.
  • [ ] Update documentation to reflect these changes and guide users accordingly.

chair28980 avatar Oct 26 '23 05:10 chair28980