go-spacemesh icon indicating copy to clipboard operation
go-spacemesh copied to clipboard

systest: add QUIC support

Open ivan4th opened this issue 10 months ago • 12 comments

Motivation

QUIC protocol is not verified in systests

Description

This adds a job that runs systests using QUIC instead of TCP

Test Plan

Make sure systests pass in both modes

TODO

  • [X] Explain motivation or link existing issue(s)
  • [x] Test changes and document test plan

ivan4th avatar Apr 29 '24 22:04 ivan4th

bors try

ivan4th avatar Apr 29 '24 22:04 ivan4th

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 80.6%. Comparing base (63dc6a8) to head (d8024a1). Report is 5 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #5902   +/-   ##
=======================================
  Coverage     80.6%   80.6%           
=======================================
  Files          286     285    -1     
  Lines        29508   29547   +39     
=======================================
+ Hits         23801   23833   +32     
- Misses        4116    4130   +14     
+ Partials      1591    1584    -7     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Apr 29 '24 23:04 codecov[bot]

try

Build failed:

spacemesh-bors[bot] avatar Apr 29 '24 23:04 spacemesh-bors[bot]

bors try

ivan4th avatar Apr 30 '24 10:04 ivan4th

try

Build failed:

spacemesh-bors[bot] avatar Apr 30 '24 10:04 spacemesh-bors[bot]

bors try

ivan4th avatar Apr 30 '24 12:04 ivan4th

try

Build failed:

spacemesh-bors[bot] avatar Apr 30 '24 12:04 spacemesh-bors[bot]

it additionally runs all of the tests but with the quic enabled? it will use twice as much resources on gke, might be just x2 cost with autoscaling.

it also seems very wastesful, as the change to run quic rather minor. some other ideas, setup half of the nodes with quic, other with tcp, run one any test with all nodes using quic, add one more additional test specifically with quic e.g TestSmeshingQuic

dshulyak avatar May 03 '24 04:05 dshulyak

Regarding adding a TestSmeshingQuic test or adapting configuration of nodes for tests, I'd say the decision should be based on what we consider the "default" setup.

If both quic and tcp are considered equally supported by us we should just make half of all nodes in every test use quic and the other half tcp. If we consider quic experimental or a feature that only some nodes should enable I'd go with a TestSmeshingQuic test.

fasmat avatar May 03 '24 08:05 fasmat

Also running all our system tests twice will more than doubles the number of tries / merges that fail due to a flaky test failing and we should look into stabilizing our system tests first before adding a bunch of additional tests.

fasmat avatar May 03 '24 08:05 fasmat

What's the status on this?

fasmat avatar Jun 04 '24 08:06 fasmat