neqo icon indicating copy to clipboard operation
neqo copied to clipboard
verified publisher icon mozilla

ci: Hook up Taskcluster

Open larseggert opened this issue 1 year ago • 6 comments

larseggert avatar May 13 '24 15:05 larseggert

Failed Interop Tests

QUIC Interop Runner, client vs. server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

github-actions[bot] avatar May 13 '24 16:05 github-actions[bot]

Benchmark results

Performance differences relative to a71e43dacf8fae41e5aa30cf95b2e826f63a7466.

  • coalesce_acked_from_zero 1+1 entries time: [197.63 ns 198.11 ns 198.61 ns] change: [+0.4613% +0.8641% +1.3272%] (p = 0.00 < 0.05) Change within noise threshold.

  • coalesce_acked_from_zero 3+1 entries time: [239.54 ns 240.16 ns 240.79 ns] change: [-0.2555% +0.0832% +0.4465%] (p = 0.65 > 0.05) No change in performance detected.

  • coalesce_acked_from_zero 10+1 entries time: [238.79 ns 239.86 ns 241.06 ns] change: [-2.5802% -0.9029% +0.2563%] (p = 0.27 > 0.05) No change in performance detected.

  • coalesce_acked_from_zero 1000+1 entries time: [219.07 ns 228.30 ns 248.61 ns] change: [-6.7599% -0.0510% +5.7818%] (p = 0.99 > 0.05) No change in performance detected.

  • RxStreamOrderer::inbound_frame() time: [119.18 ms 119.35 ms 119.60 ms] change: [-1.1253% -0.8799% -0.6225%] (p = 0.00 < 0.05) Change within noise threshold.

  • transfer/Run multiple transfers with varying seeds time: [122.07 ms 122.37 ms 122.67 ms] thrpt: [32.609 MiB/s 32.689 MiB/s 32.767 MiB/s] change: time: [+1.4442% +1.7929% +2.1576%] (p = 0.00 < 0.05) thrpt: [-2.1121% -1.7613% -1.4237%] Change within noise threshold.

  • transfer/Run multiple transfers with the same seed time: [122.20 ms 122.46 ms 122.73 ms] thrpt: [32.593 MiB/s 32.663 MiB/s 32.732 MiB/s] change: time: [+1.6190% +1.9637% +2.2814%] (p = 0.00 < 0.05) thrpt: [-2.2305% -1.9259% -1.5932%] Change within noise threshold.

  • 1-conn/1-100mb-resp (aka. Download)/client time: [1.0794 s 1.0976 s 1.1183 s] thrpt: [89.419 MiB/s 91.106 MiB/s 92.640 MiB/s] change: time: [-3.1746% +0.3112% +3.8481%] (p = 0.87 > 0.05) thrpt: [-3.7055% -0.3103% +3.2787%] No change in performance detected.

  • 1-conn/10_000-parallel-1b-resp (aka. RPS)/client time: [387.91 ms 391.40 ms 394.95 ms] thrpt: [25.320 Kelem/s 25.550 Kelem/s 25.779 Kelem/s] change: time: [-1.6162% -0.5057% +0.6474%] (p = 0.40 > 0.05) thrpt: [-0.6433% +0.5083% +1.6428%] No change in performance detected.

  • 1-conn/1-1b-resp (aka. HPS)/client time: [41.800 ms 41.901 ms 42.024 ms] thrpt: [23.796 elem/s 23.866 elem/s 23.923 elem/s] change: time: [-0.3243% +0.0889% +0.4830%] (p = 0.67 > 0.05) thrpt: [-0.4807% -0.0888% +0.3253%] No change in performance detected.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 664.9 ± 327.3 415.4 1359.7 1.00
neqo msquic reno on 949.9 ± 242.1 736.5 1314.2 1.00
neqo msquic reno 863.8 ± 116.8 783.3 1173.8 1.00
neqo msquic cubic on 840.8 ± 116.9 735.5 1069.9 1.00
neqo msquic cubic 1043.7 ± 400.8 756.1 1856.3 1.00
msquic neqo reno on 4239.5 ± 172.7 4128.6 4720.6 1.00
msquic neqo reno 4172.3 ± 20.4 4138.1 4218.0 1.00
msquic neqo cubic on 4163.7 ± 255.4 3903.2 4536.1 1.00
msquic neqo cubic 3969.1 ± 186.6 3748.8 4463.1 1.00
neqo neqo reno on 3123.4 ± 152.9 2993.3 3518.7 1.00
neqo neqo reno 3322.2 ± 210.8 3002.7 3657.4 1.00
neqo neqo cubic on 3775.6 ± 233.7 3401.4 4254.7 1.00
neqo neqo cubic 4007.6 ± 253.1 3696.3 4341.7 1.00

:arrow_down: Download logs

github-actions[bot] avatar May 13 '24 16:05 github-actions[bot]

Firefox builds for this PR

The following builds are available for testing. Crossed-out builds did not succeed.

github-actions[bot] avatar May 13 '24 18:05 github-actions[bot]

Uh oh! Looks like an error! Details

Taskcluster-GitHub attempted to cancel previously created task groups with following scopes:

assume:repo:github.com/mozilla/neqo:*, queue:seal-task-group:taskcluster-github/*, queue:cancel-task-group:taskcluster-github/*

No task-group with taskGroupId: cCMVqjy-QzO_DeA82gw26g

  • method: sealTaskGroup
  • errorCode: ResourceNotFound
  • statusCode: 404
  • time: 2024-05-14T05:57:45.309Z

firefoxci-taskcluster[bot] avatar May 14 '24 05:05 firefoxci-taskcluster[bot]

Uh oh! Looks like an error! Details

Taskcluster-GitHub attempted to cancel previously created task groups with following scopes:

assume:repo:github.com/mozilla/neqo:*, queue:seal-task-group:taskcluster-github/*, queue:cancel-task-group:taskcluster-github/*

No task-group with taskGroupId: cCMVqjy-QzO_DeA82gw26g

  • method: sealTaskGroup
  • errorCode: ResourceNotFound
  • statusCode: 404
  • time: 2024-05-15T11:52:02.616Z

firefoxci-taskcluster[bot] avatar May 15 '24 11:05 firefoxci-taskcluster[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.19%. Comparing base (e44c472) to head (cbcbc9a). Report is 2 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1901      +/-   ##
==========================================
- Coverage   93.20%   93.19%   -0.01%     
==========================================
  Files         111      110       -1     
  Lines       36025    35760     -265     
==========================================
- Hits        33578    33328     -250     
+ Misses       2447     2432      -15

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar May 15 '24 12:05 codecov[bot]

This apparently needs to exist in main to succeed, so preparing to merge.

larseggert avatar Jun 18 '24 07:06 larseggert