neqo SCONE

This turned out to be relatively easy to implement.

Does doesn't do anything with the SCONE indications that are received; as this is intended for use in a browser, there isn't an easy way to plumb those through to something that might use those indications. For the moment, these are just logged.

Jul 30 '25 06:07 martinthomson

Codecov Report

:x: Patch coverage is 95.83333% with 5 lines in your changes missing coverage. Please review. :white_check_mark: Project coverage is 93.39%. Comparing base (f0f793c) to head (8825e39). :warning: Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2814      +/-   ##
==========================================
- Coverage   93.40%   93.39%   -0.01%     
==========================================
  Files         124      124              
  Lines       36151    36218      +67     
  Branches    36151    36218      +67     
==========================================
+ Hits        33767    33827      +60     
- Misses       1538     1543       +5     
- Partials      846      848       +2

Components	Coverage Δ
neqo-common	`97.32% <ø> (ø)`
neqo-crypto	`83.25% <ø> (-0.48%)`	:arrow_down:
neqo-http3	`93.34% <ø> (ø)`
neqo-qpack	`94.18% <ø> (ø)`
neqo-transport	`94.48% <95.83%> (+0.03%)`	:arrow_up:
neqo-udp	`79.42% <ø> (ø)`
mtu	`85.76% <ø> (ø)`

Jul 30 '25 06:07 codecov[bot]

I'm still stumped on the performance hit here. Any ideas?

Jul 30 '25 10:07 martinthomson

Nothing comes to mind. I will try to reproduce locally.

Jul 30 '25 10:07 mxinden

Hypothesis:

We no longer pad with 0x00s, but 0xc8 instead.
0x00 padding is filtered out early in our processing pipeline https://github.com/mozilla/neqo/blob/4ad2f9a3a98db0b164b6106d79959831799dd84b/neqo-transport/src/packet/mod.rs#L619-L628
0xc8 is interpreted as a long packet, then later on discarded.
I assume the additional processing itself is expensive. In addition we now have a qwarn! triggering in our hot path. https://github.com/mozilla/neqo/blob/4ad2f9a3a98db0b164b6106d79959831799dd84b/neqo-transport/src/connection/mod.rs#L1498-L1500
Executing the transfer benchmark locally, stdout is spammed with the qwarn! above.

Can we special case 0xc8 in packet/mod.rs encode to discard the SCONE padding early @martinthomson?

Jul 30 '25 10:07 mxinden

We should also add a preference for this (default on).

Jul 30 '25 10:07 larseggert

I'm not sure about that hypothesis: we don't send these packets very often and an all-0xc8 packet usually fails decryption (because it's a long header packet with a connection ID longer than the remaining bytes). That's FASTER path to rejecting the packet than attempting to decrypt a short header packet.

As for the qwarn!(), we should downgrade that to qinfo!() or even qdebug!(), because dropping packets is totally normal. I'll try that out.

Jul 30 '25 11:07 martinthomson

Oh, I just realized what might be going on here:

https://github.com/mozilla/neqo/blob/4ad2f9a3a98db0b164b6106d79959831799dd84b/neqo-transport/src/packet/mod.rs#L667-L680

This might be more work than following the short header path. Maybe this needs a flag where that option produces an error instead. That code only makes sense in the very early phases of server setup.

Jul 30 '25 11:07 martinthomson

Testing on my machine shows less-than-ideal performance, hinting at a slowdown, but I can't spot the problem still:

$ target/release/deps/transfer-ea5f6c8313464190 --bench -b main
transfer/pacing-false/varying-seeds
                        time:   [20.598 ms 20.658 ms 20.723 ms]
                        change: [−0.1420% +0.3377% +0.8482%] (p = 0.18 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

transfer/pacing-true/varying-seeds
                        time:   [20.808 ms 20.894 ms 20.988 ms]
                        change: [+1.8460% +2.2808% +2.7827%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
  8 (8.00%) high mild
  3 (3.00%) high severe

transfer/pacing-false/same-seed
                        time:   [20.479 ms 20.539 ms 20.608 ms]
                        change: [+0.9765% +1.3664% +1.7723%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe

transfer/pacing-true/same-seed
                        time:   [20.728 ms 20.785 ms 20.847 ms]
                        change: [−0.0452% +0.3311% +0.6927%] (p = 0.08 > 0.05)
                        No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

Jul 31 '25 08:07 martinthomson

For what it is worth, I too am unable to reproduce the regression locally with the latest patches. Here using the Download benchmark:

critcmp main scone    
group                                         main                                   scone
-----                                         ----                                   -----
1-conn/1-100mb-resp (aka. Download)/client    1.01    299.1±6.00ms   334.3 MB/sec    1.00    297.1±3.57ms   336.6 MB/sec

Jul 31 '25 09:07 mxinden

Do the flamegraphs look different?

Jul 31 '25 10:07 larseggert

@mxinden, are you able to help out here? I know that this isn't high priority, but I really can't see how this change would materially affect performance. I'm unable to get flamegraphs from my machine easily, so your help would be appreciated.

Aug 18 '25 07:08 martinthomson

I'm unable to get flamegraphs from my machine easily, so your help would be appreciated.

You can "Download data for profiler.firefox.com" and then (with samply installed do, for example,

samply load neqo-neqo-cubic.client.perf.samply.json.gz

Aug 18 '25 08:08 larseggert

So this is now up to date with the latest in the spec. And all the performance problems are addressed (thanks Max for picking that out). We only IGNORE scone signals, so we don't need to worry about https://github.com/ietf-wg-scone/scone/pull/54 and friends. The only concern is whether we can ship this with the unallocated codepoints.

Aug 21 '25 00:08 martinthomson

Should the WG as for early IANA allocation?

Aug 21 '25 05:08 larseggert

That's the plan, though there is so much bickering over irrelevant details, that I'm reluctant to do that. (Which I guess answers the question about landing this...)

Aug 21 '25 05:08 martinthomson

Add a connection parameter so we can land and pref it off?

Aug 21 '25 06:08 larseggert

@mxinden is in the process of vendoring in a neqo release, so merge what you want to land soon.

Aug 21 '25 06:08 larseggert

Let's skip that release then. It might be better to skip a release while things settle out.

Aug 21 '25 06:08 martinthomson

Blocked by lack of final IANA codepoints.

Aug 27 '25 07:08 larseggert

@martinthomson any news re: IANA codepoints?

Oct 09 '25 12:10 larseggert

Bencher Report

Branch	scone
Testbed	On-prem

🚨 3 Alerts

Iteration	Benchmark	Measure Units	View	Benchmark Result (Result Δ%)	Upper Boundary (Limit %)
0	google vs. neqo (cubic, paced)	Latency milliseconds (ms)	📈 plot 🚷 threshold 🚨 alert (🔔)	281.26 ms (+1.25%) Baseline: 277.80 ms	280.93 ms (100.12%)
2	neqo vs. google (cubic, paced)	Latency milliseconds (ms)	📈 plot 🚷 threshold 🚨 alert (🔔)	770.01 ms (+1.61%) Baseline: 757.79 ms	766.53 ms (100.45%)
4	neqo vs. neqo (cubic)	Latency milliseconds (ms)	📈 plot 🚷 threshold 🚨 alert (🔔)	96.32 ms (+5.95%) Baseline: 90.92 ms	95.70 ms (100.65%)

Click to view all benchmark results

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
google vs. neqo (cubic, paced)	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	281.26 ms (+1.25%) Baseline: 277.80 ms	280.93 ms (100.12%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
msquic vs. neqo (cubic, paced)	📈 view plot 🚷 view threshold	202.26 ms (+2.25%) Baseline: 197.80 ms	234.78 ms (86.15%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. google (cubic, paced)	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	770.01 ms (+1.61%) Baseline: 757.79 ms	766.53 ms (100.45%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. msquic (cubic, paced)	📈 view plot 🚷 view threshold	159.72 ms (+1.23%) Baseline: 157.78 ms	160.78 ms (99.34%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. neqo (cubic)	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	96.32 ms (+5.95%) Baseline: 90.92 ms	95.70 ms (100.65%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. neqo (cubic, paced)	📈 view plot 🚷 view threshold	96.76 ms (+4.81%) Baseline: 92.32 ms	97.06 ms (99.69%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. neqo (reno)	📈 view plot 🚷 view threshold	94.80 ms (+4.18%) Baseline: 90.99 ms	95.81 ms (98.95%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. neqo (reno, paced)	📈 view plot 🚷 view threshold	96.69 ms (+4.82%) Baseline: 92.25 ms	96.90 ms (99.78%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. quiche (cubic, paced)	📈 view plot 🚷 view threshold	193.63 ms (-0.04%) Baseline: 193.71 ms	197.11 ms (98.24%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
neqo vs. s2n (cubic, paced)	📈 view plot 🚷 view threshold	223.81 ms (+1.28%) Baseline: 220.98 ms	223.89 ms (99.97%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
quiche vs. neqo (cubic, paced)	📈 view plot 🚷 view threshold	154.09 ms (+0.68%) Baseline: 153.05 ms	158.41 ms (97.27%)

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
s2n vs. neqo (cubic, paced)	📈 view plot 🚷 view threshold	173.37 ms (-0.44%) Baseline: 174.14 ms	178.20 ms (97.29%)

🐰 View full continuous benchmarking report in Bencher

Nov 02 '25 15:11 github-actions[bot]

Bencher Report

Branch	scone
Testbed	On-prem

Click to view all benchmark results

Benchmark	Latency	Benchmark Result nanoseconds (ns) (Result Δ%)	Upper Boundary nanoseconds (ns) (Limit %)
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client	📈 view plot 🚷 view threshold	202,590,000.00 ns (-2.09%) Baseline: 206,915,535.71 ns	217,159,737.21 ns (93.29%)
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client	📈 view plot 🚷 view threshold	198,750,000.00 ns (-1.08%) Baseline: 200,919,678.57 ns	211,977,656.90 ns (93.76%)
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client	📈 view plot 🚷 view threshold	38,614,000.00 ns (+28.65%) Baseline: 30,014,553.57 ns	38,834,114.80 ns (99.43%)
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client	📈 view plot 🚷 view threshold	285,700,000.00 ns (-2.18%) Baseline: 292,070,250.00 ns	304,889,871.10 ns (93.71%)
1-streams/each-1000-bytes/simulated-time	📈 view plot 🚷 view threshold	118,970,000.00 ns (+0.32%) Baseline: 118,588,857.14 ns	120,800,869.77 ns (98.48%)
1-streams/each-1000-bytes/wallclock-time	📈 view plot 🚷 view threshold	589,410.00 ns (-0.79%) Baseline: 594,113.93 ns	618,038.86 ns (95.37%)
1000-streams/each-1-bytes/simulated-time	📈 view plot 🚷 view threshold	14,994,000,000.00 ns (+0.00%) Baseline: 14,993,371,428.57 ns	15,011,666,803.43 ns (99.88%)
1000-streams/each-1-bytes/wallclock-time	📈 view plot 🚷 view threshold	13,606,000.00 ns (-2.97%) Baseline: 14,022,064.29 ns	14,893,959.05 ns (91.35%)
1000-streams/each-1000-bytes/simulated-time	📈 view plot 🚷 view threshold	18,943,000,000.00 ns (-0.08%) Baseline: 18,958,367,857.14 ns	19,233,360,057.78 ns (98.49%)
1000-streams/each-1000-bytes/wallclock-time	📈 view plot 🚷 view threshold	47,697,000.00 ns (-6.27%) Baseline: 50,887,767.86 ns	57,642,611.62 ns (82.75%)
RxStreamOrderer::inbound_frame()	📈 view plot 🚷 view threshold	109,100,000.00 ns (-0.50%) Baseline: 109,643,642.86 ns	111,661,047.44 ns (97.71%)
coalesce_acked_from_zero 1+1 entries	📈 view plot 🚷 view threshold	89.29 ns (+0.70%) Baseline: 88.67 ns	89.53 ns (99.74%)
coalesce_acked_from_zero 10+1 entries	📈 view plot 🚷 view threshold	106.10 ns (+0.06%) Baseline: 106.03 ns	107.11 ns (99.06%)
coalesce_acked_from_zero 1000+1 entries	📈 view plot 🚷 view threshold	91.37 ns (+1.65%) Baseline: 89.88 ns	94.19 ns (97.01%)
coalesce_acked_from_zero 3+1 entries	📈 view plot 🚷 view threshold	106.23 ns (-0.30%) Baseline: 106.55 ns	107.60 ns (98.73%)
decode 1048576 bytes, mask 3f	📈 view plot 🚷 view threshold	1,589,400.00 ns (-2.45%) Baseline: 1,629,356.07 ns	1,796,685.47 ns (88.46%)
decode 1048576 bytes, mask 7f	📈 view plot 🚷 view threshold	5,053,000.00 ns (-0.27%) Baseline: 5,066,498.57 ns	5,112,482.30 ns (98.84%)
decode 1048576 bytes, mask ff	📈 view plot 🚷 view threshold	3,034,400.00 ns (+0.18%) Baseline: 3,028,805.36 ns	3,047,003.29 ns (99.59%)
decode 4096 bytes, mask 3f	📈 view plot 🚷 view threshold	5,541.20 ns (-26.99%) Baseline: 7,589.43 ns	10,399.47 ns (53.28%)
decode 4096 bytes, mask 7f	📈 view plot 🚷 view threshold	19,456.00 ns (-1.99%) Baseline: 19,851.86 ns	20,473.58 ns (95.03%)
decode 4096 bytes, mask ff	📈 view plot 🚷 view threshold	10,670.00 ns (-6.75%) Baseline: 11,441.97 ns	12,557.73 ns (84.97%)
sent::Packets::take_ranges	📈 view plot 🚷 view threshold	4,546.10 ns (-3.96%) Baseline: 4,733.55 ns	4,972.28 ns (91.43%)
transfer/pacing-false/same-seed/simulated-time/run	📈 view plot 🚷 view threshold	25,710,000,000.00 ns (+1.14%) Baseline: 25,418,956,834.53 ns	26,071,176,299.93 ns (98.61%)
transfer/pacing-false/same-seed/wallclock-time/run	📈 view plot 🚷 view threshold	25,061,000.00 ns (-3.14%) Baseline: 25,874,615.11 ns	27,028,139.12 ns (92.72%)
transfer/pacing-false/varying-seeds/simulated-time/run	📈 view plot 🚷 view threshold	25,172,000,000.00 ns (-0.00%) Baseline: 25,172,341,726.62 ns	25,220,118,696.55 ns (99.81%)
transfer/pacing-false/varying-seeds/wallclock-time/run	📈 view plot 🚷 view threshold	24,513,000.00 ns (-5.48%) Baseline: 25,934,906.47 ns	27,500,645.35 ns (89.14%)
transfer/pacing-true/same-seed/simulated-time/run	📈 view plot 🚷 view threshold	25,675,000,000.00 ns (+0.18%) Baseline: 25,629,622,302.16 ns	25,731,312,433.86 ns (99.78%)
transfer/pacing-true/same-seed/wallclock-time/run	📈 view plot 🚷 view threshold	25,803,000.00 ns (-5.06%) Baseline: 27,177,485.61 ns	28,686,876.26 ns (89.95%)
transfer/pacing-true/varying-seeds/simulated-time/run	📈 view plot 🚷 view threshold	24,954,000,000.00 ns (-0.16%) Baseline: 24,993,672,661.87 ns	25,042,687,291.64 ns (99.65%)
transfer/pacing-true/varying-seeds/wallclock-time/run	📈 view plot 🚷 view threshold	24,942,000.00 ns (-5.67%) Baseline: 26,441,928.06 ns	28,087,481.03 ns (88.80%)

🐰 View full continuous benchmarking report in Bencher

Nov 02 '25 15:11 github-actions[bot]

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest vs. go-x-net: :warning:BP BA
neqo-latest vs. haproxy: :warning:BP BA
neqo-latest vs. kwik: :warning:S L1 C1 BP BA
neqo-latest vs. linuxquic: :warning:R C1
neqo-latest vs. lsquic: run cancelled after 20 min
neqo-latest vs. msquic: :warning:R Z A L1 C1
neqo-latest vs. mvfst: :warning:H DC LR M R Z 3 B U A L1 L2 C1 C2 6 BP BA
neqo-latest vs. neqo: :warning:A
neqo-latest vs. neqo-latest: :warning:A
neqo-latest vs. nginx: :warning:BP BA
neqo-latest vs. ngtcp2: :warning:CM
neqo-latest vs. picoquic: :warning:A L1
neqo-latest vs. quic-go: :warning:A
neqo-latest vs. quiche: :warning:C1 BP BA
neqo-latest vs. quinn: :warning:A
neqo-latest vs. s2n-quic: :warning:BA CM
neqo-latest vs. tquic: :warning:S BP BA
neqo-latest vs. xquic: :warning:A L1 C1

neqo-latest as server

aioquic vs. neqo-latest: :warning:C1 CM
go-x-net vs. neqo-latest: :warning:CM
kwik vs. neqo-latest: :warning:BP BA CM
lsquic vs. neqo-latest: :warning:BA
msquic vs. neqo-latest: :warning:U CM
mvfst vs. neqo-latest: :warning:Z A L1 C1 CM
neqo vs. neqo-latest: :warning:A
openssl vs. neqo-latest: :warning:LR M A CM
quic-go vs. neqo-latest: :warning:CM
quiche vs. neqo-latest: :warning:CM
quinn vs. neqo-latest: :warning:V2 CM
s2n-quic vs. neqo-latest: :warning:CM
tquic vs. neqo-latest: :warning:CM
xquic vs. neqo-latest: :warning:M CM

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest vs. aioquic: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2 BP BA~~
neqo-latest vs. go-x-net: :rocket:~~H DC LR M B U A L2 C2 6~~
neqo-latest vs. haproxy: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2~~
neqo-latest vs. kwik: :rocket:~~H DC LR C20 M R Z 3 B U A L2 C2 6 V2~~
neqo-latest vs. linuxquic: :rocket:~~H DC LR C20 M S Z 3 B U E A L1 L2 C2 6 V2 BP BA CM~~
neqo-latest vs. msquic: :rocket:~~H DC LR C20 M S B U L2 C2 6 V2 BP BA~~
neqo-latest vs. neqo: :rocket:~~H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM~~
neqo-latest vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM~~
neqo-latest vs. nginx: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6~~
neqo-latest vs. ngtcp2: :rocket:~~H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA~~
neqo-latest vs. picoquic: :rocket:~~H DC LR C20 M S R Z 3 B U E L2 C1 C2 6 V2 BP BA~~
neqo-latest vs. quic-go: :rocket:~~H DC LR C20 M S R Z 3 B U L1 L2 C1 C2 6 BP BA~~
neqo-latest vs. quiche: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C2 6~~
neqo-latest vs. quinn: :rocket:~~H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 BP BA~~
neqo-latest vs. s2n-quic: :rocket:~~H DC LR C20 M S R 3 B U E A L1 L2 C1 C2 6 BP~~
neqo-latest vs. tquic: :rocket:~~H DC LR C20 M R Z 3 B U A L1 L2 C1 C2 6~~
neqo-latest vs. xquic: :rocket:~~H DC LR C20 M S R Z 3 B U L2 C2 6 BP BA~~

neqo-latest as server

aioquic vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C2 6 V2 BP BA~~
chrome vs. neqo-latest: :rocket:~~3~~
go-x-net vs. neqo-latest: :rocket:~~H DC LR M B U A L2 C2 6 BP BA~~
kwik vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2~~
linuxquic vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM~~
lsquic vs. neqo-latest: :rocket:~~H DC LR C20 M S R 3 B E A L1 L2 C1 C2 6 V2 BP CM~~
msquic vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z B A L1 L2 C1 C2 6 V2 BP BA~~
mvfst vs. neqo-latest: :rocket:~~H DC LR M 3 B L2 C2 6 BP BA~~
neqo vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM~~
ngtcp2 vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM~~
openssl vs. neqo-latest: :rocket:~~H DC C20 S R 3 B L2 C2 6 BP BA~~
picoquic vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM~~
quic-go vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 BP BA~~
quiche vs. neqo-latest: :rocket:~~H DC LR M S R Z 3 B A L1 L2 C1 C2 6 BP BA~~
quinn vs. neqo-latest: :rocket:~~H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 BP BA~~
s2n-quic vs. neqo-latest: :rocket:~~H DC LR M S R 3 B E A L1 L2 C1 C2 6 BP BA~~
tquic vs. neqo-latest: :rocket:~~H DC LR M S R Z 3 B A L1 L2 C1 C2 6 BP BA~~
xquic vs. neqo-latest: :rocket:~~H DC LR C20 S R Z 3 B U A L1 L2 C1 C2 6 BP BA~~

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest vs. aioquic: E CM
neqo-latest vs. go-x-net: C20 S R Z 3 E L1 C1 V2 CM
neqo-latest vs. haproxy: E CM
neqo-latest vs. kwik: E CM
neqo-latest vs. msquic: 3 E CM
neqo-latest vs. mvfst: C20 S E V2 CM
neqo-latest vs. nginx: E V2 CM
neqo-latest vs. picoquic: CM
neqo-latest vs. quic-go: E V2 CM
neqo-latest vs. quiche: E V2 CM
neqo-latest vs. quinn: V2 CM
neqo-latest vs. s2n-quic: Z V2
neqo-latest vs. tquic: E V2 CM
neqo-latest vs. xquic: E V2 CM

neqo-latest as server

aioquic vs. neqo-latest: E
chrome vs. neqo-latest: H DC LR C20 M S R Z B U E A L1 L2 C1 C2 6 V2 BP BA CM
go-x-net vs. neqo-latest: C20 S R Z 3 E L1 C1 V2
kwik vs. neqo-latest: E
lsquic vs. neqo-latest: Z U
msquic vs. neqo-latest: 3 E
mvfst vs. neqo-latest: C20 S R U E V2
openssl vs. neqo-latest: Z U E L1 C1 V2
quic-go vs. neqo-latest: E V2
quiche vs. neqo-latest: C20 U E V2
s2n-quic vs. neqo-latest: C20 Z U V2
tquic vs. neqo-latest: C20 U E V2
xquic vs. neqo-latest: E V2

Nov 02 '25 17:11 github-actions[bot]

Client/server transfer results

Performance differences relative to aa1451274c811495215dcfa478ca8b8c58f6fe5c.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params)	Mean ± σ	Min	Max	MiB/s ± σ	Δ `main`	Δ `main`
google vs. google	462.2 ± 3.6	456.3	472.2	69.2 ± 8.9
google vs. neqo (cubic, paced)	281.3 ± 4.8	272.8	291.5	113.8 ± 6.7	:green_heart: -1.6	-0.6%
msquic vs. msquic	180.6 ± 73.1	136.8	559.6	177.2 ± 0.4
msquic vs. neqo (cubic, paced)	202.3 ± 72.9	149.9	660.6	158.2 ± 0.4	-6.3	-3.0%
neqo vs. google (cubic, paced)	770.0 ± 5.2	762.6	788.3	41.6 ± 6.2	:broken_heart: 2.3	0.3%
neqo vs. msquic (cubic, paced)	159.7 ± 7.4	151.6	187.6	200.3 ± 4.3	0.6	0.4%
neqo vs. neqo (cubic)	96.3 ± 5.2	86.0	111.1	332.2 ± 6.2	1.4	1.4%
neqo vs. neqo (cubic, paced)	96.8 ± 6.5	87.9	124.3	330.7 ± 4.9	1.1	1.1%
neqo vs. neqo (reno)	94.8 ± 4.7	86.1	107.7	337.6 ± 6.8	-0.1	-0.1%
neqo vs. neqo (reno, paced)	96.7 ± 6.7	86.3	121.9	330.9 ± 4.8	0.5	0.6%
neqo vs. quiche (cubic, paced)	193.6 ± 5.3	185.7	211.9	165.3 ± 6.0	0.3	0.2%
neqo vs. s2n (cubic, paced)	223.8 ± 5.2	214.8	239.3	143.0 ± 6.2	-0.2	-0.1%
quiche vs. neqo (cubic, paced)	154.1 ± 5.8	141.9	176.8	207.7 ± 5.5	:broken_heart: 1.5	1.0%
quiche vs. quiche	146.5 ± 5.1	136.2	156.3	218.5 ± 6.3
s2n vs. neqo (cubic, paced)	173.4 ± 4.5	163.8	183.1	184.6 ± 7.1	0.3	0.2%
s2n vs. s2n	251.7 ± 26.4	235.4	349.7	127.2 ± 1.2

Download data for profiler.firefox.com or download performance comparison data.

Nov 02 '25 17:11 github-actions[bot]

Benchmark results

Performance differences relative to aa1451274c811495215dcfa478ca8b8c58f6fe5c.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: Change within noise threshold.

       time:   [198.40 ms 198.75 ms 199.10 ms]
       thrpt:  [502.25 MiB/s 503.16 MiB/s 504.02 MiB/s]
change:
       time:   [−1.2174% −0.9593% −0.7045%] (p = 0.00 +0.9686% +1.2324%]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.

       time:   [283.86 ms 285.70 ms 287.57 ms]
       thrpt:  [34.774 Kelem/s 35.001 Kelem/s 35.228 Kelem/s]
change:
       time:   [−0.8559% +0.0412% +0.9754%] (p = 0.93 > 0.05)
       thrpt:  [−0.9659% −0.0411% +0.8633%]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.

       time:   [38.479 ms 38.614 ms 38.768 ms]
       thrpt:  [25.795   B/s 25.898   B/s 25.988   B/s]
change:
       time:   [−0.8408% −0.2423% +0.3184%] (p = 0.42 > 0.05)
       thrpt:  [−0.3174% +0.2429% +0.8479%]
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe

1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: :green_heart: Performance has improved.

       time:   [202.35 ms 202.59 ms 202.84 ms]
       thrpt:  [492.99 MiB/s 493.60 MiB/s 494.18 MiB/s]
change:
       time:   [−2.7926% −2.6126% −2.4428%] (p = 0.00 +2.6827% +2.8729%]
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe

decode 4096 bytes, mask ff: No change in performance detected.

       time:   [10.624 µs 10.670 µs 10.726 µs]
       change: [−0.8798% −0.2914% +0.2748%] (p = 0.34 > 0.05)
Found 15 outliers among 100 measurements (15.00%)
6 (6.00%) low mild
9 (9.00%) high severe

decode 1048576 bytes, mask ff: No change in performance detected.

       time:   [3.0242 ms 3.0344 ms 3.0460 ms]
       change: [−1.3588% −0.4120% +0.3471%] (p = 0.38 > 0.05)
Found 11 outliers among 100 measurements (11.00%)
11 (11.00%) high severe

decode 4096 bytes, mask 7f: No change in performance detected.

       time:   [19.378 µs 19.456 µs 19.546 µs]
       change: [−0.6583% −0.0149% +0.5632%] (p = 0.96 > 0.05)
Found 18 outliers among 100 measurements (18.00%)
3 (3.00%) low mild
1 (1.00%) high mild
14 (14.00%) high severe

decode 1048576 bytes, mask 7f: No change in performance detected.

       time:   [5.0382 ms 5.0530 ms 5.0705 ms]
       change: [−0.6033% −0.1432% +0.3401%] (p = 0.56 > 0.05)
Found 14 outliers among 100 measurements (14.00%)
14 (14.00%) high severe

decode 4096 bytes, mask 3f: No change in performance detected.

       time:   [5.5250 µs 5.5412 µs 5.5648 µs]
       change: [−0.1868% +0.5250% +1.5387%] (p = 0.34 > 0.05)
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) low mild
1 (1.00%) high mild
5 (5.00%) high severe

decode 1048576 bytes, mask 3f: No change in performance detected.

       time:   [1.5851 ms 1.5894 ms 1.5950 ms]
       change: [−0.6041% −0.0839% +0.3625%] (p = 0.78 > 0.05)
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe

1-streams/each-1000-bytes/wallclock-time: Change within noise threshold.

       time:   [587.46 µs 589.41 µs 591.65 µs]
       change: [+0.4912% +0.9613% +1.4458%] (p = 0.00 Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) high severe
1-streams/each-1000-bytes/simulated-time
time:   [118.76 ms 118.97 ms 119.18 ms]
thrpt:  [8.1938 KiB/s 8.2082 KiB/s 8.2227 KiB/s]
change:
time:   [−0.2099% +0.0342% +0.2966%] (p = 0.79 > 0.05)
thrpt:  [−0.2957% −0.0342% +0.2103%]
No change in performance detected.

1000-streams/each-1-bytes/wallclock-time: No change in performance detected.

       time:   [13.584 ms 13.606 ms 13.629 ms]
       change: [−0.2886% −0.0353% +0.2246%] (p = 0.79 > 0.05)
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
1000-streams/each-1-bytes/simulated-time
time:   [14.979 s 14.994 s 15.008 s]
thrpt:  [66.632   B/s 66.695   B/s 66.758   B/s]
change:
time:   [−0.0954% +0.0437% +0.1750%] (p = 0.53 > 0.05)
thrpt:  [−0.1747% −0.0437% +0.0955%]
No change in performance detected.

1000-streams/each-1000-bytes/wallclock-time: No change in performance detected.

       time:   [47.533 ms 47.697 ms 47.860 ms]
       change: [−1.0661% −0.3703% +0.2445%] (p = 0.28 > 0.05)
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
1000-streams/each-1000-bytes/simulated-time
time:   [18.756 s 18.943 s 19.128 s]
thrpt:  [51.054 KiB/s 51.553 KiB/s 52.066 KiB/s]
change:
time:   [−1.9875% −0.6874% +0.5932%] (p = 0.31 > 0.05)
thrpt:  [−0.5897% +0.6922% +2.0278%]
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild

coalesce_acked_from_zero 1+1 entries: No change in performance detected.

       time:   [88.985 ns 89.294 ns 89.609 ns]
       change: [−0.3636% +0.1896% +0.9283%] (p = 0.60 > 0.05)
Found 9 outliers among 100 measurements (9.00%)
6 (6.00%) high mild
3 (3.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.

       time:   [105.89 ns 106.23 ns 106.58 ns]
       change: [−0.6321% −0.0640% +0.4745%] (p = 0.83 > 0.05)
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) high mild
9 (9.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.

       time:   [105.58 ns 106.10 ns 106.67 ns]
       change: [−0.2122% +0.2093% +0.5967%] (p = 0.33 > 0.05)
Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low severe
3 (3.00%) low mild
2 (2.00%) high mild
8 (8.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.

       time:   [91.261 ns 91.367 ns 91.490 ns]
       change: [−9.4895% −3.3462% +0.3179%] (p = 0.34 > 0.05)
Found 21 outliers among 100 measurements (21.00%)
7 (7.00%) high mild
14 (14.00%) high severe

RxStreamOrderer::inbound_frame(): :green_heart: Performance has improved.

       time:   [109.03 ms 109.10 ms 109.18 ms]
       change: [−1.5679% −1.2886% −1.0867%] (p = 0.00 Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe

sent::Packets::take_ranges: No change in performance detected.

       time:   [4.4557 µs 4.5461 µs 4.6301 µs]
       change: [−3.4371% −0.6095% +2.1997%] (p = 0.69 > 0.05)
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-false/varying-seeds/wallclock-time/run: Change within noise threshold.

       time:   [24.481 ms 24.513 ms 24.547 ms]
       change: [−1.0294% −0.8269% −0.6241%] (p = 0.00 Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-false/varying-seeds/simulated-time/run: No change in performance detected.

       time:   [25.133 s 25.172 s 25.212 s]
       thrpt:  [162.46 KiB/s 162.72 KiB/s 162.97 KiB/s]
change:
       time:   [−0.3343% −0.1357% +0.0745%] (p = 0.21 > 0.05)
       thrpt:  [−0.0745% +0.1359% +0.3355%]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-true/varying-seeds/wallclock-time/run: Change within noise threshold.

       time:   [24.884 ms 24.942 ms 25.004 ms]
       change: [−1.7648% −1.4439% −1.1139%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe

transfer/pacing-true/varying-seeds/simulated-time/run: No change in performance detected.

       time:   [24.917 s 24.954 s 24.990 s]
       thrpt:  [163.90 KiB/s 164.14 KiB/s 164.38 KiB/s]
change:
       time:   [−0.2640% −0.0399% +0.1802%] (p = 0.71 > 0.05)
       thrpt:  [−0.1799% +0.0399% +0.2647%]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-false/same-seed/wallclock-time/run: No change in performance detected.

       time:   [25.027 ms 25.061 ms 25.106 ms]
       change: [−0.2881% −0.0423% +0.1850%] (p = 0.75 > 0.05)
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe

transfer/pacing-false/same-seed/simulated-time/run: No change in performance detected.

       time:   [25.710 s 25.710 s 25.710 s]
       thrpt:  [159.31 KiB/s 159.31 KiB/s 159.31 KiB/s]
change:
       time:   [+0.0000% +0.0000% +0.0000%] (p = NaN > 0.05)
       thrpt:  [+0.0000% +0.0000% +0.0000%]

transfer/pacing-true/same-seed/wallclock-time/run: Change within noise threshold.

       time:   [25.781 ms 25.803 ms 25.830 ms]
       change: [−1.6376% −1.4443% −1.2802%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe

transfer/pacing-true/same-seed/simulated-time/run: No change in performance detected.

       time:   [25.675 s 25.675 s 25.675 s]
       thrpt:  [159.53 KiB/s 159.53 KiB/s 159.53 KiB/s]
change:
       time:   [+0.0000% +0.0000% +0.0000%] (p = NaN > 0.05)
       thrpt:  [+0.0000% +0.0000% +0.0000%]

Download data for profiler.firefox.com or download performance comparison data.

Nov 02 '25 18:11 github-actions[bot]