12/WAKU2-FILTER: Production readiness

[jm-clius]

Background

The 12/WAKU2-FILTER protocol and specification have a number of outstanding questions that need to be addressed to reach production readiness.

1. We deal badly with failed nodes (on both the client and server side) and cleaning up stale connections:

• Filter will keep attempting to push messages indefinitely until a client node unsubscribes or a node restarts, even if the node client goes offline. Something more sophisticated is required, so the nodes are able to drop the peer once it’s determined that a client node is unreachable (either time based or via a limit on the total number of subscriptions)
• Client nodes have no way of knowing if a full node failed: https://github.com/vacp2p/rfc/issues/253

2. The protocol negotiation may require a rethink. Specifically, the response to a filter request should be an ACK/NACK that the request has been accepted. Currently the filtered messages are pushed in repeated responses. Related issue: https://github.com/vacp2p/rfc/issues/207
3. The use of RequestId is inconsistent: https://github.com/status-im/nim-waku/issues/257

cc @richard-ramos

[rshiv]

I am looking at the stale connection issue and focusing on the client failure case as this issue will have a higher impact on the filter node memory. A proposed solution is discussed in #475 . After addressing will have more clear version of the WAKU2-FILTER to address server failure.

The 2nd and 3rd issue relates to design and would need a exhaustive discussion to progress further which will be on the pipeline after addressing the 1st.

[jm-clius]

Furthermore, most current implementations are not designed with browser clients in mind. See: https://github.com/status-im/nwaku/issues/948

[jm-clius]

Fixed with new filter version: https://rfc.vac.dev/spec/12/