js-waku icon indicating copy to clipboard operation
js-waku copied to clipboard

test: create scripts for running light-push/filter and measuring ratio of messages sent and received

Open adklempner opened this issue 1 year ago • 4 comments

This is a bug report/feature request/support request/change request

Problem

Current efforts of testing long-term usage of js-waku required manually running an example and observing it. We should move towards a more automated solution.

Proposed Solutions

  1. Script that once loaded in an html page (or headless browser) begins sending monotonically increasing messages over light push. It should clearly display/report which messages failed to send a. parametrize period between messages b. parameterize whether to use default bootstrapping or specific nodes/fleets c. parameterize content topic/shard info
  2. Script that once loaded in an html page (or headless browser) listens to message sent from 1. It should clearly display/report which message were missed.

Notes

  • Refer to browser testing package

adklempner avatar May 09 '24 17:05 adklempner

Sounds good. I am guessing we should aim to use different nodes for filter and light push? Caveat is if the node is offline, then it may not be straight forward to detect, I guess that light push would fail which will help detect it? We need to be sure we cater for scenarios with total loss of connections and understand how we measure those effectively.

fryorcraken avatar May 21 '24 01:05 fryorcraken

The scope of this work has changed since I started and discussed more with others, I've captured it here: https://github.com/waku-org/lab.waku.org/issues/69

adklempner avatar May 21 '24 05:05 adklempner

Our strategy is to:

  • run pre-release https://github.com/waku-org/js-waku/actions/workflows/pre-release.yml;
  • run lab.waku.org/dogfooding;
  • check browser logs for errors / misbehaviors;
  • see telemetry and derive any insight;
  • share within the team;

This is absolutely necessary and assumption is that it is part of any change we do now.

cc @waku-org/js-waku @waku-org/js-waku-developers

weboko avatar Sep 17 '24 13:09 weboko

As for making it part of automated CI we have to investigate how it can be incorporated into PR gates as there is a risk of making it too flaky.

weboko avatar Sep 17 '24 13:09 weboko

To be addressed in https://github.com/waku-org/js-waku/issues/2217

weboko avatar Mar 24 '25 22:03 weboko