js-waku
js-waku copied to clipboard
test: create scripts for running light-push/filter and measuring ratio of messages sent and received
This is a bug report/feature request/support request/change request
Problem
Current efforts of testing long-term usage of js-waku required manually running an example and observing it. We should move towards a more automated solution.
Proposed Solutions
- Script that once loaded in an html page (or headless browser) begins sending monotonically increasing messages over light push. It should clearly display/report which messages failed to send a. parametrize period between messages b. parameterize whether to use default bootstrapping or specific nodes/fleets c. parameterize content topic/shard info
- Script that once loaded in an html page (or headless browser) listens to message sent from 1. It should clearly display/report which message were missed.
Notes
- Refer to browser testing package
Sounds good. I am guessing we should aim to use different nodes for filter and light push? Caveat is if the node is offline, then it may not be straight forward to detect, I guess that light push would fail which will help detect it? We need to be sure we cater for scenarios with total loss of connections and understand how we measure those effectively.
The scope of this work has changed since I started and discussed more with others, I've captured it here: https://github.com/waku-org/lab.waku.org/issues/69
Our strategy is to:
- run pre-release https://github.com/waku-org/js-waku/actions/workflows/pre-release.yml;
- run
lab.waku.org/dogfooding; - check browser logs for errors / misbehaviors;
- see telemetry and derive any insight;
- share within the team;
This is absolutely necessary and assumption is that it is part of any change we do now.
cc @waku-org/js-waku @waku-org/js-waku-developers
As for making it part of automated CI we have to investigate how it can be incorporated into PR gates as there is a risk of making it too flaky.
To be addressed in https://github.com/waku-org/js-waku/issues/2217