neqo icon indicating copy to clipboard operation
neqo copied to clipboard

Published 20 hours ago • verified publisher icon mozilla

fix(transport/ecn): start after handshake and path validation

Open mxinden opened this issue 8 months ago • 2 comments

Fixes https://github.com/mozilla/neqo/issues/2490.

Only start ECN marking and thus ECN-Path validation when:

  • Connection handshake is confirmed.
  • AND, for any secondary path, path is validated (e.g. via PATH_RESPONSE).

Draft for now. Opening up early to prevent duplicate work.

mxinden avatar Apr 04 '25 19:04 mxinden

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to 0bd3933b2ccfed140b903cc76e7a7f8c1daa3c56.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

  • neqo-latest vs. aioquic: H :warning:DC LR C20 S R Z B U A L1 :rocket:~~L2 C1~~ 6 :warning:V2 :rocket:~~BP BA~~
  • neqo-latest vs. go-x-net: :warning:H LR :rocket:~~DC M~~ B U A :warning:L2 C2 6
  • neqo-latest vs. haproxy: :warning:DC :rocket:~~H~~ LR :warning:M S R Z :warning:3 U A :warning:L2 C1 C2 :rocket:~~6~~ V2
  • neqo-latest vs. kwik: :rocket:~~H DC~~ LR C20 M :warning:S R Z 3 B :warning:L2 :rocket:~~U A C2~~ 6 V2
  • neqo-latest vs. linuxquic: H DC LR C20 M R Z :rocket:~~3~~ B :warning:E A L2 :warning:6 BA :rocket:~~C2 V2 BP~~ CM
  • neqo-latest vs. lsquic: H LR C20 :rocket:~~M~~ S R Z 3 B :warning:U E A :rocket:~~C2~~ 6 :warning:V2 BP :warning:BA :rocket:~~CM~~
  • neqo-latest vs. msquic: :warning:H DC S :rocket:~~LR C20 M~~ C2 6 :warning:V2 :rocket:~~BP~~ BA
  • neqo-latest vs. mvfst: :warning:LR :rocket:~~H DC~~ M R 3 :warning:U :rocket:~~B~~ L2 :warning:C1 C2 6 :warning:BP BA
  • neqo-latest vs. neqo: H DC LR C20 M S :warning:R Z 3 B :warning:U E :warning:L1 C2 6 V2 BP :rocket:~~L2 C1~~ BA :rocket:~~CM~~
  • neqo-latest vs. neqo-latest: H DC :warning:M S :rocket:~~LR C20~~ Z 3 B :rocket:~~U~~ E L1 :warning:C2 :rocket:~~C1~~ 6 :warning:V2 BA :rocket:~~BP~~ CM
  • neqo-latest vs. nginx: :warning:LR :rocket:~~H~~ S :warning:3 B U A L1 C1 :rocket:~~R 6~~
  • neqo-latest vs. ngtcp2: H :warning:DC LR C20 M :warning:R Z :rocket:~~S~~ 3 B U :warning:E A :warning:C1 C2 6 :rocket:~~L1 L2~~ V2 BP :warning:BA
  • neqo-latest vs. picoquic: :warning:H DC LR :rocket:~~C20~~ M :warning:B E :rocket:~~S R Z 3 U~~ L2 :warning:6 :rocket:~~C2~~ V2 :rocket:~~BA~~
  • neqo-latest vs. quic-go: :rocket:~~DC LR~~ C20 :warning:M S :rocket:~~R~~ Z :rocket:~~3~~ B :rocket:~~U L1 L2~~ C1 C2 6 BP
  • neqo-latest vs. quiche: DC :rocket:~~LR~~ C20 :warning:M S R Z :warning:3 B U A L1 L2 C1 :rocket:~~C2~~ 6
  • neqo-latest vs. quinn: H DC :warning:C20 S :rocket:~~LR M~~ B U E L2 :rocket:~~C1~~ C2 6 BP :rocket:~~BA~~
  • neqo-latest vs. s2n-quic: H :warning:DC LR C20 M :warning:S 3 B U :warning:A L2 :warning:C2 :rocket:~~C1~~ 6 BP
  • neqo-latest vs. tquic: :warning:DC C20 :rocket:~~H LR~~ M R Z 3 B :warning:U A L1 L2 C1 :rocket:~~C2~~ 6
  • neqo-latest vs. xquic: H DC :rocket:~~LR~~ C20 M Z :warning:3 B U L2 C2 6 :rocket:~~BP BA~~

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

github-actions[bot] avatar Apr 04 '25 19:04 github-actions[bot]

Benchmark results

Performance differences relative to 3909abe17a4f9a6e8d85fd0a0d0b80571042c88c.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: Change within noise threshold.
       time:   [653.17 ms 654.62 ms 656.13 ms]
       thrpt:  [152.41 MiB/s 152.76 MiB/s 153.10 MiB/s]
change:
       time:   [−0.6433% −0.3870% −0.1110%] (p = 0.00 +0.3885% +0.6475%]

Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [305.14 ms 306.57 ms 307.97 ms]
       thrpt:  [32.471 Kelem/s 32.619 Kelem/s 32.771 Kelem/s]
change:
       time:   [−0.3048% +0.3810% +1.0538%] (p = 0.27 > 0.05)
       thrpt:  [−1.0428% −0.3796% +0.3058%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [26.913 ms 27.053 ms 27.213 ms]
       thrpt:  [36.748  elem/s 36.965  elem/s 37.156  elem/s]
change:
       time:   [−0.4503% +0.2610% +1.0196%] (p = 0.47 > 0.05)
       thrpt:  [−1.0093% −0.2603% +0.4523%]

Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high severe
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: Change within noise threshold.
       time:   [660.83 ms 662.31 ms 663.80 ms]
       thrpt:  [150.65 MiB/s 150.99 MiB/s 151.32 MiB/s]
change:
       time:   [−0.6393% −0.3896% −0.1148%] (p = 0.00 +0.3911% +0.6435%]
decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.818 µs 11.864 µs 11.914 µs]
       change: [−0.0598% +0.2597% +0.6180%] (p = 0.13 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
3 (3.00%) low mild
1 (1.00%) high mild
11 (11.00%) high severe
decode 1048576 bytes, mask ff: No change in performance detected.
       time:   [3.0218 ms 3.0312 ms 3.0423 ms]
       change: [−0.4463% +0.0386% +0.5230%] (p = 0.89 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
9 (9.00%) high severe
decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [19.922 µs 19.964 µs 20.014 µs]
       change: [−0.5146% −0.0800% +0.4248%] (p = 0.75 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
2 (2.00%) low severe
5 (5.00%) low mild
1 (1.00%) high mild
9 (9.00%) high severe
decode 1048576 bytes, mask 7f: No change in performance detected.
       time:   [5.0444 ms 5.0556 ms 5.0685 ms]
       change: [−0.4599% −0.0952% +0.2405%] (p = 0.60 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
12 (12.00%) high severe
decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [8.2596 µs 8.2896 µs 8.3259 µs]
       change: [−0.3200% +0.0738% +0.5247%] (p = 0.75 > 0.05)

Found 23 outliers among 100 measurements (23.00%)
2 (2.00%) low severe
6 (6.00%) low mild
3 (3.00%) high mild
12 (12.00%) high severe
decode 1048576 bytes, mask 3f: No change in performance detected.
       time:   [1.5849 ms 1.5905 ms 1.5973 ms]
       change: [−0.6282% −0.1014% +0.4344%] (p = 0.70 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
1000 streams of 1 bytes/multistream: :broken_heart: Performance has regressed.
       time:   [50.923 ns 51.040 ns 51.164 ns]
       change: [+55.287% +56.773% +58.245%] (p = 0.00 Found 31 outliers among 500 measurements (6.20%)
2 (0.40%) low severe
9 (1.80%) low mild
3 (0.60%) high mild
17 (3.40%) high severe
1000 streams of 1000 bytes/multistream: :broken_heart: Performance has regressed.
       time:   [51.241 ns 51.379 ns 51.525 ns]
       change: [+61.845% +63.311% +64.761%] (p = 0.00 Found 43 outliers among 500 measurements (8.60%)
1 (0.20%) low severe
5 (1.00%) low mild
13 (2.60%) high mild
24 (4.80%) high severe
Criterion.rs ERROR: Error in Gnuplot:          line 0: Can't plot with an empty x range!
: No change in performance detected.
coalesce_acked_from_zero 1+1 entries
time:   [88.271 ns 88.555 ns 88.835 ns]
change: [−0.4836% +0.2282% +1.2638%] (p = 0.66 > 0.05)


Found 9 outliers among 100 measurements (9.00%)
8 (8.00%) high mild
1 (1.00%) high severe
coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [106.10 ns 106.51 ns 106.93 ns]
       change: [−0.4317% +1.0524% +3.5603%] (p = 0.54 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
1 (1.00%) low mild
4 (4.00%) high mild
12 (12.00%) high severe
coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [105.25 ns 105.48 ns 105.82 ns]
       change: [−0.4960% +0.0515% +0.6013%] (p = 0.86 > 0.05)

Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
4 (4.00%) high severe
coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [88.708 ns 92.298 ns 100.45 ns]
       change: [−0.5587% +2.6888% +8.3111%] (p = 0.37 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.03 ms 111.10 ms 111.17 ms]
       change: [+0.9502% +1.0311% +1.1131%] (p = 0.00 Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) low mild
sent::Packets::take_ranges: No change in performance detected.
       time:   [8.3071 µs 8.5386 µs 8.7568 µs]
       change: [−0.1886% +6.0934% +16.006%] (p = 0.15 > 0.05)

Found 21 outliers among 100 measurements (21.00%)
5 (5.00%) low severe
11 (11.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
transfer/pacing-false/varying-seeds: :green_heart: Performance has improved.
       time:   [34.603 ms 34.664 ms 34.724 ms]
       change: [−3.5790% −3.3105% −3.0545%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [36.047 ms 36.139 ms 36.231 ms]
       change: [−1.8788% −1.5171% −1.1519%] (p = 0.00 Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [35.061 ms 35.111 ms 35.161 ms]
       change: [−2.4981% −2.2964% −2.1088%] (p = 0.00 Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) low mild
1 (1.00%) high mild
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [36.492 ms 36.556 ms 36.621 ms]
       change: [−2.4463% −2.2170% −1.9969%] (p = 0.00

Client/server transfer results

Performance differences relative to 3909abe17a4f9a6e8d85fd0a0d0b80571042c88c.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params) Mean ± σ Min Max MiB/s ± σ Δ main Δ main
google vs. google 528.9 ± 32.0 506.5 734.1 60.5 ± 1.0
google vs. neqo (cubic, paced) 389.9 ± 36.2 358.8 626.6 82.1 ± 0.9 -2.1 -0.5%
msquic vs. msquic 150.0 ± 19.6 135.3 230.0 213.3 ± 1.6
msquic vs. neqo (cubic, paced) 302.6 ± 54.4 271.3 618.5 105.8 ± 0.6 -3.7 -1.2%
neqo vs. google (cubic, paced) 808.7 ± 18.5 768.5 955.1 39.6 ± 1.7 -0.7 -0.1%
neqo vs. msquic (cubic, paced) 210.3 ± 14.6 194.0 258.7 152.2 ± 2.2 -8.3 -3.8%
neqo vs. neqo (cubic) 263.0 ± 12.3 241.9 296.5 121.7 ± 2.6 2.4 0.9%
neqo vs. neqo (cubic, paced) 273.2 ± 60.9 249.7 832.3 117.2 ± 0.5 :broken_heart: 18.0 7.0%
neqo vs. neqo (reno) 261.1 ± 11.2 244.1 288.2 122.6 ± 2.9 -0.5 -0.2%
neqo vs. neqo (reno, paced) 269.8 ± 35.9 240.9 493.3 118.6 ± 0.9 :broken_heart: 10.9 4.2%
neqo vs. quiche (cubic, paced) 245.6 ± 12.5 230.6 294.5 130.3 ± 2.6 :green_heart: -13.2 -5.1%
neqo vs. s2n (cubic, paced) 263.0 ± 36.4 228.4 491.1 121.7 ± 0.9 7.6 3.0%
quiche vs. neqo (cubic, paced) 413.6 ± 37.4 377.6 611.7 77.4 ± 0.9 8.7 2.1%
quiche vs. quiche 203.0 ± 48.9 175.1 558.9 157.6 ± 0.7
s2n vs. neqo (cubic, paced) 314.0 ± 20.6 286.8 394.5 101.9 ± 1.6 -9.0 -2.8%
s2n vs. s2n 255.9 ± 31.3 232.4 422.5 125.1 ± 1.0

Download data for profiler.firefox.com or download performance comparison data.

github-actions[bot] avatar Apr 17 '25 20:04 github-actions[bot]

🐰 Bencher Report

Branchecn-after-handshake
Testbedt-linux64-ms-280

🚨 1 Alert

BenchmarkMeasure
Units		ViewBenchmark Result
(Result Δ%)		Upper Boundary
(Limit %)
coalesce_acked_from_zero 1000+1 entriesLatency
nanoseconds (ns)		📈 plot
🚷 threshold
🚨 alert (🔔)		92.30 ns
(+3.32%)Baseline: 89.33 ns
91.78 ns
(100.56%)

Click to view all benchmark results
BenchmarkLatencyBenchmark Result
nanoseconds (ns)
(Result Δ%)		Upper Boundary
nanoseconds (ns)
(Limit %)
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client📈 view plot
🚷 view threshold		662,310,000.00 ns
(-0.33%)Baseline: 664,484,459.46 ns
731,705,113.47 ns
(90.52%)
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client📈 view plot
🚷 view threshold		654,620,000.00 ns
(+3.58%)Baseline: 631,997,027.03 ns
832,542,305.02 ns
(78.63%)
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client📈 view plot
🚷 view threshold		27,053,000.00 ns
(-0.50%)Baseline: 27,189,675.68 ns
27,661,578.21 ns
(97.80%)
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client📈 view plot
🚷 view threshold		306,570,000.00 ns
(+0.57%)Baseline: 304,834,189.19 ns
316,066,323.62 ns
(97.00%)
1000 streams of 1 bytes/multistream📈 view plot
🚷 view threshold		51.04 ns
(+35.41%)Baseline: 37.69 ns
54.95 ns
(92.89%)
1000 streams of 1000 bytes/multistream📈 view plot
🚷 view threshold		51.38 ns
(+37.96%)Baseline: 37.24 ns
54.62 ns
(94.07%)
RxStreamOrderer::inbound_frame()📈 view plot
🚷 view threshold		111,100,000.00 ns
(+0.55%)Baseline: 110,495,959.46 ns
114,561,444.98 ns
(96.98%)
coalesce_acked_from_zero 1+1 entries📈 view plot
🚷 view threshold		88.56 ns
(-0.15%)Baseline: 88.69 ns
89.30 ns
(99.17%)
coalesce_acked_from_zero 10+1 entries📈 view plot
🚷 view threshold		105.48 ns
(-0.43%)Baseline: 105.93 ns
106.88 ns
(98.69%)
coalesce_acked_from_zero 1000+1 entries📈 view plot
🚷 view threshold
🚨 view alert (🔔)		92.30 ns
(+3.32%)Baseline: 89.33 ns
91.78 ns
(100.56%)

coalesce_acked_from_zero 3+1 entries📈 view plot
🚷 view threshold		106.51 ns
(-0.02%)Baseline: 106.53 ns
107.40 ns
(99.17%)
decode 1048576 bytes, mask 3f📈 view plot
🚷 view threshold		1,590,500.00 ns
(-1.60%)Baseline: 1,616,313.51 ns
1,764,082.52 ns
(90.16%)
decode 1048576 bytes, mask 7f📈 view plot
🚷 view threshold		5,055,600.00 ns
(-0.16%)Baseline: 5,063,533.78 ns
5,090,572.93 ns
(99.31%)
decode 1048576 bytes, mask ff📈 view plot
🚷 view threshold		3,031,200.00 ns
(-0.14%)Baseline: 3,035,586.49 ns
3,067,625.58 ns
(98.81%)
decode 4096 bytes, mask 3f📈 view plot
🚷 view threshold		8,289.60 ns
(+4.60%)Baseline: 7,924.70 ns
10,163.54 ns
(81.56%)
decode 4096 bytes, mask 7f📈 view plot
🚷 view threshold		19,964.00 ns
(+0.22%)Baseline: 19,919.30 ns
20,405.80 ns
(97.83%)
decode 4096 bytes, mask ff📈 view plot
🚷 view threshold		11,864.00 ns
(+0.40%)Baseline: 11,816.19 ns
11,975.56 ns
(99.07%)
sent::Packets::take_ranges📈 view plot
🚷 view threshold		8,538.60 ns
(+1.16%)Baseline: 8,440.78 ns
8,591.80 ns
(99.38%)
transfer/pacing-false/same-seed📈 view plot
🚷 view threshold		35,111,000.00 ns
(+0.53%)Baseline: 34,926,486.49 ns
36,611,004.30 ns
(95.90%)
transfer/pacing-false/varying-seeds📈 view plot
🚷 view threshold		34,664,000.00 ns
(-1.15%)Baseline: 35,066,513.51 ns
36,805,390.21 ns
(94.18%)
transfer/pacing-true/same-seed📈 view plot
🚷 view threshold		36,556,000.00 ns
(+0.06%)Baseline: 36,534,000.00 ns
38,140,269.54 ns
(95.85%)
transfer/pacing-true/varying-seeds📈 view plot
🚷 view threshold		36,139,000.00 ns
(+0.65%)Baseline: 35,906,418.92 ns
37,523,651.23 ns
(96.31%)
🐰 View full continuous benchmarking report in Bencher

github-actions[bot] avatar Jun 17 '25 09:06 github-actions[bot]

🐰 Bencher Report

Branchecn-after-handshake
Testbedt-linux64-ms-280
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
milliseconds (ms)
(Result Δ%)		Upper Boundary
milliseconds (ms)
(Limit %)
s2n vs. neqo (cubic, paced)📈 view plot
🚷 view threshold		310.42 ms
(-0.45%)Baseline: 311.83 ms
327.11 ms
(94.90%)
🐰 View full continuous benchmarking report in Bencher

github-actions[bot] avatar Jun 17 '25 09:06 github-actions[bot]

@mxinden I tried to "help" and I think I messed up the rebase...

larseggert avatar Jul 10 '25 14:07 larseggert

No worries. Rebased on to latest main now.

mxinden avatar Jul 11 '25 13:07 mxinden

Client/server transfer results

Performance differences relative to 531cb0b6c21d27ff1ccec5502ecc346a6a922d71.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params) Mean ± σ Min Max MiB/s ± σ Δ main Δ main
google vs. google 451.0 ± 4.8 443.9 462.0 70.9 ± 6.7
google vs. neqo (cubic, paced) 270.3 ± 4.7 261.9 288.0 118.4 ± 6.8 -0.7 -0.3%
msquic vs. msquic 124.0 ± 13.0 109.9 178.0 258.0 ± 2.5
msquic vs. neqo (cubic, paced) 150.9 ± 34.3 120.8 374.9 212.0 ± 0.9 4.6 3.1%
neqo vs. google (cubic, paced) 752.1 ± 4.6 744.6 764.5 42.5 ± 7.0 0.4 0.1%
neqo vs. msquic (cubic, paced) 156.2 ± 5.1 148.2 175.6 204.9 ± 6.3 -0.2 -0.1%
neqo vs. neqo (cubic) 92.2 ± 4.9 82.9 110.9 347.0 ± 6.5 :broken_heart: 2.3 2.5%
neqo vs. neqo (cubic, paced) 94.8 ± 4.2 85.4 107.8 337.4 ± 7.6 :broken_heart: 3.8 4.2%
neqo vs. neqo (reno) 91.0 ± 4.9 80.8 111.8 351.6 ± 6.5 0.9 1.0%
neqo vs. neqo (reno, paced) 92.2 ± 4.3 85.5 101.3 347.1 ± 7.4 :broken_heart: 2.1 2.3%
neqo vs. quiche (cubic, paced) 195.1 ± 4.6 186.9 204.9 164.1 ± 7.0 0.7 0.3%
neqo vs. s2n (cubic, paced) 217.7 ± 4.9 211.5 235.3 147.0 ± 6.5 -0.4 -0.2%
quiche vs. neqo (cubic, paced) 160.3 ± 4.5 149.2 169.9 199.6 ± 7.1 :broken_heart: 3.4 2.1%
quiche vs. quiche 146.5 ± 5.2 137.4 163.6 218.4 ± 6.2
s2n vs. neqo (cubic, paced) 169.4 ± 4.6 161.3 185.1 188.9 ± 7.0 :green_heart: -1.8 -1.0%
s2n vs. s2n 249.8 ± 30.5 229.4 347.8 128.1 ± 1.0

Download data for profiler.firefox.com or download performance comparison data.

github-actions[bot] avatar Jul 11 '25 13:07 github-actions[bot]

Benchmark results

Performance differences relative to 531cb0b6c21d27ff1ccec5502ecc346a6a922d71.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: Change within noise threshold.
       time:   [199.38 ms 199.81 ms 200.29 ms]
       thrpt:  [499.27 MiB/s 500.47 MiB/s 501.55 MiB/s]
change:
       time:   [+0.4936% +0.7774% +1.0872%] (p = 0.00 −0.7714% −0.4912%]

Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [303.36 ms 304.87 ms 306.37 ms]
       thrpt:  [32.640 Kelem/s 32.801 Kelem/s 32.964 Kelem/s]
change:
       time:   [−0.5614% +0.1751% +0.8849%] (p = 0.63 > 0.05)
       thrpt:  [−0.8771% −0.1748% +0.5646%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [27.455 ms 27.546 ms 27.660 ms]
       thrpt:  [36.153   B/s 36.303   B/s 36.424   B/s]
change:
       time:   [−0.1785% +0.2806% +0.7581%] (p = 0.26 > 0.05)
       thrpt:  [−0.7524% −0.2799% +0.1788%]

Found 19 outliers among 100 measurements (19.00%)
1 (1.00%) low mild
18 (18.00%) high severe
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: :green_heart: Performance has improved.
       time:   [614.57 ms 619.13 ms 622.98 ms]
       thrpt:  [160.52 MiB/s 161.52 MiB/s 162.72 MiB/s]
change:
       time:   [−4.4209% −3.3279% −2.2884%] (p = 0.00 +3.4424% +4.6254%]

Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) low severe
2 (2.00%) low mild
1 (1.00%) high severe
decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.813 µs 11.845 µs 11.885 µs]
       change: [−0.1456% +0.3796% +1.1232%] (p = 0.27 > 0.05)

Found 16 outliers among 100 measurements (16.00%)
3 (3.00%) low severe
4 (4.00%) low mild
1 (1.00%) high mild
8 (8.00%) high severe
decode 1048576 bytes, mask ff: No change in performance detected.
       time:   [3.0206 ms 3.0301 ms 3.0412 ms]
       change: [−0.7263% −0.1769% +0.3167%] (p = 0.53 > 0.05)

Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) high mild
9 (9.00%) high severe
decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [20.003 µs 20.051 µs 20.106 µs]
       change: [−0.0550% +0.4063% +1.0006%] (p = 0.13 > 0.05)

Found 17 outliers among 100 measurements (17.00%)
3 (3.00%) low severe
1 (1.00%) low mild
13 (13.00%) high severe
decode 1048576 bytes, mask 7f: No change in performance detected.
       time:   [5.0432 ms 5.0585 ms 5.0769 ms]
       change: [−0.2586% +0.1203% +0.5741%] (p = 0.58 > 0.05)

Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low mild
14 (14.00%) high severe
decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [8.2880 µs 8.3257 µs 8.3692 µs]
       change: [−0.4558% +0.3830% +1.1278%] (p = 0.37 > 0.05)

Found 23 outliers among 100 measurements (23.00%)
9 (9.00%) low mild
2 (2.00%) high mild
12 (12.00%) high severe
decode 1048576 bytes, mask 3f: No change in performance detected.
       time:   [1.5852 ms 1.5908 ms 1.5976 ms]
       change: [−0.7821% −0.1839% +0.4184%] (p = 0.53 > 0.05)

Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) high mild
5 (5.00%) high severe
1000 streams of 1 bytes/multistream: No change in performance detected.
       time:   [27.109 ns 27.307 ns 27.516 ns]
       change: [−34.524% −14.331% +1.7113%] (p = 0.60 > 0.05)

Found 75 outliers among 500 measurements (15.00%)
60 (12.00%) high mild
15 (3.00%) high severe
Criterion.rs ERROR: Error in Gnuplot:          line 0: Can't plot with an empty x range!
: 
Criterion.rs ERROR: Error in Gnuplot:          line 0: Can't plot with an empty x range!
: No change in performance detected.
1000 streams of 1000 bytes/multistream
time:   [27.389 ns 31.185 ns 38.586 ns]
change: [+0.8046% +15.030% +54.391%] (p = 0.21 > 0.05)


Found 38 outliers among 500 measurements (7.60%)
27 (5.40%) high mild
11 (2.20%) high severe
coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [88.160 ns 88.528 ns 88.893 ns]
       change: [−0.3781% +0.1738% +0.7427%] (p = 0.55 > 0.05)

Found 12 outliers among 100 measurements (12.00%)
9 (9.00%) high mild
3 (3.00%) high severe
coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [105.48 ns 105.83 ns 106.22 ns]
       change: [−0.6046% −0.1401% +0.3511%] (p = 0.57 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
3 (3.00%) high mild
11 (11.00%) high severe
coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [104.84 ns 105.19 ns 105.63 ns]
       change: [−0.3236% +0.2731% +0.9357%] (p = 0.40 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) low mild
3 (3.00%) high mild
5 (5.00%) high severe
coalesce_acked_from_zero 1000+1 entries: Change within noise threshold.
       time:   [88.548 ns 88.715 ns 88.921 ns]
       change: [−1.5290% −0.7327% −0.0641%] (p = 0.05 Found 10 outliers among 100 measurements (10.00%)
3 (3.00%) high mild
7 (7.00%) high severe
RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [108.44 ms 108.57 ms 108.74 ms]
       change: [−1.2441% −1.0937% −0.9305%] (p = 0.00 Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
2 (2.00%) high mild
2 (2.00%) high severe
sent::Packets::take_ranges: No change in performance detected.
       time:   [8.0682 µs 8.3085 µs 8.5428 µs]
       change: [−1.9447% +4.0692% +14.590%] (p = 0.40 > 0.05)

Found 23 outliers among 100 measurements (23.00%)
4 (4.00%) low severe
13 (13.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
transfer/pacing-false/varying-seeds: Change within noise threshold.
       time:   [37.289 ms 37.367 ms 37.446 ms]
       change: [+0.8769% +1.2026% +1.5124%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [38.069 ms 38.191 ms 38.323 ms]
       change: [+0.5182% +0.9683% +1.4091%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [36.904 ms 36.988 ms 37.087 ms]
       change: [+0.5750% +0.8624% +1.1783%] (p = 0.00 Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [38.733 ms 38.816 ms 38.901 ms]
       change: [+0.1120% +0.4055% +0.6994%] (p = 0.01 Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

Download data for profiler.firefox.com or download performance comparison data.

github-actions[bot] avatar Jul 11 '25 13:07 github-actions[bot]

~~Only 10 more tests to go:~~

failures:
    connection::tests::keys::discarded_initial_keys
    connection::tests::migration::migrate_same_fail
    connection::tests::priority::critical
    connection::tests::priority::high_normal
    connection::tests::priority::important
    connection::tests::recovery::pto_handshake_complete
    connection::tests::resumption::ticket_rtt_larger_than_default
    connection::tests::resumption::ticket_rtt_less_than_default
    connection::tests::vn::compatible_upgrade
    connection::tests::vn::compatible_upgrade_large_initial

mxinden avatar Jul 11 '25 20:07 mxinden

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.89%. Comparing base (531cb0b) to head (aa968f9).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #2560      +/-   ##
==========================================
- Coverage   94.92%   94.89%   -0.04%     
==========================================
  Files         115      115              
  Lines       34281    34286       +5     
  Branches    34281    34286       +5     
==========================================
- Hits        32541    32534       -7     
- Misses       1733     1745      +12     
  Partials        7        7
Components Coverage Δ
neqo-common 97.10% <ø> (ø)
neqo-crypto 89.64% <ø> (ø)
neqo-http3 93.71% <ø> (ø)
neqo-qpack 95.45% <ø> (ø)
neqo-transport 95.95% <100.00%> (-0.07%) :arrow_down:
neqo-udp 89.37% <ø> (-0.49%) :arrow_down:

codecov[bot] avatar Jul 12 '25 08:07 codecov[bot]