go-waku
go-waku copied to clipboard
waku-org
feat: ping cache
Description
Cache that contains the ping times for peers, for reusing the value instead of pinging a peer each time you want to select the fastest peer. Ping times are verified every 10 seconds.
Jenkins Builds
Click to see older builds (11)
| :grey_question: | Commit | :hash: | Finished (UTC) | Duration | Platform | Result |
|---|---|---|---|---|---|---|
| :heavy_check_mark: | 316074e6 | #1 | 2024-01-05 19:04:09 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | d03e4bd9 | #2 | 2024-01-05 19:31:32 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | 7d6a0a96 | #3 | 2024-01-08 22:50:13 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | db8c2a2d | #4 | 2024-01-08 22:52:09 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | 924a595f | #5 | 2024-01-08 22:54:10 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | 9432653c | #6 | 2024-01-10 20:40:23 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | 724064e5 | #7 | 2024-01-10 20:46:38 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | a289c4c1 | #8 | 2024-01-11 22:14:42 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | 72ceaf34 | #9 | 2024-01-11 22:16:38 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | da20aef7 | #10 | 2024-01-11 22:19:10 | ~1 min | nix-flake |
:page_facing_up:log |
| :heavy_multiplication_x: | ab3169e4 | #11 | 2024-05-10 08:58:39 | ~2 min | nix-flake |
:page_facing_up:log |
| :grey_question: | Commit | :hash: | Finished (UTC) | Duration | Platform | Result |
|---|---|---|---|---|---|---|
| :heavy_multiplication_x: | be9f1b89 | #12 | 2024-05-13 18:11:26 | ~54 sec | nix-flake |
:page_facing_up:log |
| :heavy_check_mark: | fda5263f | #13 | 2024-05-13 18:28:23 | ~1 min | nix-flake |
:page_facing_up:log |
Another thought comes to my mind is since messages would anyways be sent/received from Filter node either via FilterPing or MessagePush. Can we somehow also use those messages RTT (from transport layer) and in that case Ping would only get used in idle periods.
Makes me wonder if we are creating lot of traffic in the network just for RTT maintenance. Need to do some more study regarding if this is something that libp2p provides.
Can we somehow also use those messages RTT (from transport layer) and in that case Ping would only get used in idle periods.
Would be nice if libp2p had something to let us know how long it took to open a stream. I did try to find this unsuccesfully tho :(
@chaitanyaprem, I found something interesting in yamux!!:
Looks like it measures RTT for all the open sessions:
- https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L164C1-L164C24
- https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L340
meaning that instead of doing a Ping ourselves, we can use
Peerstore().LatencyEWMA(peer.ID) time.Durationto obtain an exponentially-weighted moving avg. I think we can use this, since we're dropping support mplex! (we would still need to ping those peers that do not have a latency since it means we have never pinged them before).
There's also a keepAlive with a configurable interval: https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L430C1-L430C37 which i seems interesting too because maybe I can use this instead of the keepAlive code in go-waku to handle session connectivity
Seems to me that this should simplify the code quite a bit :)
@chaitanyaprem, I found something interesting in yamux!!:
Looks like it measures RTT for all the open sessions:
* https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L164C1-L164C24 * https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L340 meaning that instead of doing a Ping ourselves, we can use `Peerstore().LatencyEWMA(peer.ID) time.Duration` to obtain an exponentially-weighted moving avg. I think we can use this, since we're dropping support mplex! (we would still need to ping those peers that do not have a latency since it means we have never pinged them before).There's also a keepAlive with a configurable interval: https://github.com/libp2p/go-yamux/blob/48aa3a7bb46c354076969f5b3ac7f4b805744dcb/session.go#L430C1-L430C37 which i seems interesting too because maybe I can use this instead of the keepAlive code in go-waku to handle session connectivity
Seems to me that this should simplify the code quite a bit :)
That's awesome, definitely will simply a lot of overhead at our layer. And since mplex is being dropped, it makes sense to use this.
I did a new version using the latency. It is more efficient than the current peer selector on master branch because it uses the latency metric recorded by yamux, but still does a Ping() for those peers which have no latency recorded.
when can it be merged? version v0.9.0 will be got error because libp2p has removed muxer/mplex package
Yamux is currently not used in nwaku. Let's wait for this PR to be merged https://github.com/waku-org/nwaku/pull/2397 and for the waku/status fleets to be updated with a version that includes this change.
@richard-ramos , time to merge this PR i guess. CI issues need to be fixed though It would be useful for peer management for status.