neqo icon indicating copy to clipboard operation
neqo copied to clipboard
verified publisher icon mozilla

Introduce Certificate Compression Zlib Encoding/Decoding

Open Frosne opened this issue 8 months ago • 5 comments

This patch:

  1. Extends bindings to support certificate compression set functionality and some functions to support it
  2. Adds to flate2 to provide encoders/decoders (already present in https://searchfox.org/mozilla-central/source/third_party/rust/flate2, thus this specific implementation)
  3. Provide a function set_zlib_certificate_compression that will enable advertising certificate compression in clientHello and decoding using zlib if we received a zlib encoded certificate
  4. Introduce a test to check if connection is successful when both parties use zlib (important: server is encoding the certificate in this case) and only client supports zlib

Frosne avatar Apr 09 '25 14:04 Frosne

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to c1679d89b5c3192429ff1bc56ed21e10dc699eef.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

github-actions[bot] avatar Apr 09 '25 17:04 github-actions[bot]

Benchmark results

Performance differences relative to c1679d89b5c3192429ff1bc56ed21e10dc699eef.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected.
       time:   [731.23 ms 735.37 ms 739.47 ms]
       thrpt:  [135.23 MiB/s 135.99 MiB/s 136.76 MiB/s]
change:
       time:   [-0.4332% +0.3781% +1.1959%] (p = 0.37 > 0.05)
       thrpt:  [-1.1818% -0.3767% +0.4351%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [351.12 ms 352.77 ms 354.43 ms]
       thrpt:  [28.214 Kelem/s 28.347 Kelem/s 28.480 Kelem/s]
change:
       time:   [-0.3657% +0.3107% +0.9876%] (p = 0.37 > 0.05)
       thrpt:  [-0.9779% -0.3097% +0.3671%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [24.952 ms 25.102 ms 25.257 ms]
       thrpt:  [39.594  elem/s 39.838  elem/s 40.078  elem/s]
change:
       time:   [-1.0446% -0.1804% +0.7256%] (p = 0.69 > 0.05)
       thrpt:  [-0.7204% +0.1808% +1.0557%]
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: :green_heart: Performance has improved.
       time:   [1.8023 s 1.8214 s 1.8420 s]
       thrpt:  [54.289 MiB/s 54.904 MiB/s 55.486 MiB/s]
change:
       time:   [-6.8089% -5.2273% -3.6394%] (p = 0.00 +5.5157% +7.3064%]

Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
decode 4096 bytes, mask ff: Change within noise threshold.
       time:   [12.001 µs 12.037 µs 12.081 µs]
       change: [-0.8513% -0.4616% -0.0633%] (p = 0.03 Found 12 outliers among 100 measurements (12.00%)
2 (2.00%) low severe
1 (1.00%) low mild
9 (9.00%) high severe
decode 1048576 bytes, mask ff: :green_heart: Performance has improved.
       time:   [2.9598 ms 2.9716 ms 2.9849 ms]
       change: [-5.9212% -5.4027% -4.8453%] (p = 0.00 Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low mild
14 (14.00%) high severe
decode 4096 bytes, mask 7f: Change within noise threshold.
       time:   [20.002 µs 20.053 µs 20.110 µs]
       change: [-1.1784% -0.8268% -0.4312%] (p = 0.00 Found 18 outliers among 100 measurements (18.00%)
4 (4.00%) low severe
1 (1.00%) low mild
1 (1.00%) high mild
12 (12.00%) high severe
decode 1048576 bytes, mask 7f: :green_heart: Performance has improved.
       time:   [4.7891 ms 4.8003 ms 4.8132 ms]
       change: [-8.9311% -8.6384% -8.3110%] (p = 0.00 Found 12 outliers among 100 measurements (12.00%)
12 (12.00%) high severe
decode 4096 bytes, mask 3f: :green_heart: Performance has improved.
       time:   [6.3171 µs 6.3417 µs 6.3731 µs]
       change: [-10.246% -9.6553% -8.8812%] (p = 0.00 Found 17 outliers among 100 measurements (17.00%)
3 (3.00%) low severe
3 (3.00%) low mild
4 (4.00%) high mild
7 (7.00%) high severe
decode 1048576 bytes, mask 3f: :broken_heart: Performance has regressed.
       time:   [2.1492 ms 2.1560 ms 2.1631 ms]
       change: [+19.308% +19.898% +20.467%] (p = 0.00 Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
4 (4.00%) high mild
7 (7.00%) high severe
1 streams of 1 bytes/multistream: Change within noise threshold.
       time:   [72.303 µs 72.542 µs 72.787 µs]
       change: [-1.5786% -1.1323% -0.6670%] (p = 0.00 Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1000 streams of 1 bytes/multistream: :broken_heart: Performance has regressed.
       time:   [25.382 ms 25.418 ms 25.455 ms]
       change: [+2.1935% +2.4147% +2.6257%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
10000 streams of 1 bytes/multistream: Change within noise threshold.
       time:   [1.6991 s 1.7009 s 1.7027 s]
       change: [+0.7265% +0.8839% +1.0290%] (p = 0.00 Found 19 outliers among 100 measurements (19.00%)
10 (10.00%) low mild
6 (6.00%) high mild
3 (3.00%) high severe
1 streams of 1000 bytes/multistream: No change in performance detected.
       time:   [73.721 µs 74.391 µs 75.507 µs]
       change: [-2.6289% -0.7800% +1.1622%] (p = 0.49 > 0.05)

Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
100 streams of 1000 bytes/multistream: No change in performance detected.
       time:   [3.3718 ms 3.3788 ms 3.3863 ms]
       change: [-0.1775% +0.1220% +0.4262%] (p = 0.42 > 0.05)

Found 21 outliers among 100 measurements (21.00%)
21 (21.00%) high severe
1000 streams of 1000 bytes/multistream: Change within noise threshold.
       time:   [143.17 ms 143.24 ms 143.32 ms]
       change: [+0.2676% +0.3417% +0.4145%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [94.898 ns 95.275 ns 95.638 ns]
       change: [-0.7177% -0.1690% +0.3524%] (p = 0.54 > 0.05)

Found 14 outliers among 100 measurements (14.00%)
12 (12.00%) high mild
2 (2.00%) high severe
coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [112.67 ns 112.95 ns 113.25 ns]
       change: [-0.1580% +0.3799% +1.0512%] (p = 0.26 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) low mild
1 (1.00%) high mild
11 (11.00%) high severe
coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [111.95 ns 112.40 ns 113.03 ns]
       change: [-0.8708% -0.2912% +0.2511%] (p = 0.33 > 0.05)

Found 16 outliers among 100 measurements (16.00%)
5 (5.00%) low severe
1 (1.00%) low mild
4 (4.00%) high mild
6 (6.00%) high severe
coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [93.277 ns 98.283 ns 108.86 ns]
       change: [-1.0282% +1.9021% +6.8645%] (p = 0.44 > 0.05)

Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [115.78 ms 115.83 ms 115.88 ms]
       change: [+0.7897% +0.8501% +0.9144%] (p = 0.00 Found 18 outliers among 100 measurements (18.00%)
2 (2.00%) low severe
5 (5.00%) low mild
11 (11.00%) high mild
SentPackets::take_ranges: No change in performance detected.
       time:   [8.2557 µs 8.5202 µs 8.7681 µs]
       change: [-4.4384% -1.7111% +1.0751%] (p = 0.24 > 0.05)

Found 21 outliers among 100 measurements (21.00%)
1 (1.00%) low severe
17 (17.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
transfer/pacing-false/varying-seeds: Change within noise threshold.
       time:   [35.900 ms 35.971 ms 36.043 ms]
       change: [-1.1506% -0.8643% -0.5771%] (p = 0.00 Found 3 outliers among 100 measurements (3.00%)
1 (1.00%) low mild
2 (2.00%) high mild
transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [36.846 ms 36.956 ms 37.065 ms]
       change: [-1.1956% -0.7908% -0.3701%] (p = 0.00
transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [35.517 ms 35.560 ms 35.602 ms]
       change: [-0.9195% -0.7589% -0.5880%] (p = 0.00 Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
transfer/pacing-true/same-seed: No change in performance detected.
       time:   [37.346 ms 37.414 ms 37.481 ms]
       change: [-0.4140% -0.1642% +0.0806%] (p = 0.21 > 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild

Client/server transfer results

Performance differences relative to c1679d89b5c3192429ff1bc56ed21e10dc699eef.

Transfer of 33554432 bytes over loopback, 30 runs. All unit-less numbers are in milliseconds.

Client Server CC Pacing Mean ± σ Min Max MiB/s ± σ Δ main Δ main
neqo neqo reno on 323.3 ± 31.1 297.9 458.3 99.0 ± 1.0 1.6 0.5%
neqo neqo reno 343.8 ± 74.9 292.1 591.4 93.1 ± 0.4 -20.5 -5.6%
neqo neqo cubic on 344.2 ± 86.7 300.1 776.4 93.0 ± 0.4 13.7 4.2%
neqo neqo cubic 327.0 ± 37.2 295.4 458.5 97.9 ± 0.9 2.0 0.6%
google neqo reno on 769.7 ± 88.8 546.5 958.0 41.6 ± 0.4 5.9 0.8%
google neqo reno 768.7 ± 94.7 554.5 988.3 41.6 ± 0.3 2.9 0.4%
google neqo cubic on 764.5 ± 80.3 575.3 878.4 41.9 ± 0.4 3.0 0.4%
google neqo cubic 767.1 ± 90.6 573.0 958.7 41.7 ± 0.4 2.0 0.3%
google google 578.3 ± 41.8 552.8 777.2 55.3 ± 0.8 7.0 1.2%
neqo msquic reno on 275.9 ± 42.6 244.8 422.4 116.0 ± 0.8 8.8 3.3%
neqo msquic reno 267.1 ± 25.9 243.5 371.2 119.8 ± 1.2 -1.0 -0.4%
neqo msquic cubic on 268.9 ± 29.7 237.9 408.2 119.0 ± 1.1 -2.0 -0.7%
neqo msquic cubic 270.3 ± 32.8 245.3 424.9 118.4 ± 1.0 3.0 1.1%
msquic msquic 179.6 ± 25.4 145.6 248.9 178.2 ± 1.3 -0.2 -0.1%

:arrow_down: Download logs

github-actions[bot] avatar Apr 09 '25 17:04 github-actions[bot]

I'm not sure that this is the right way to build this. In NSS, we don't include the compressor. That's so that the integration is left to the using code, especially Gecko, so that Gecko can reuse the (de)compressor that it has. Having neqo bring its own (de)compressor denies the application that flexibility.

I would prefer to see a plugin-based approach, similar to the one used by NSS.

Do I understand correctly that you suggest to provide only the interface? I.e. instead of set_zlib_certificate_compression we gonna have set_certificate_compression that takes as an input the SSLCertificateCompressionAlgorithm object?

https://searchfox.org/mozilla-central/source/security/nss/lib/ssl/sslexp.h#1082

Frosne avatar Apr 10 '25 09:04 Frosne

Right. I imagine you would define a trait that defines the compression interface. Then the code would hold a Box<dyn TheTrait>.

martinthomson avatar Apr 11 '25 16:04 martinthomson

Right. I imagine you would define a trait that defines the compression interface. Then the code would hold a Box<dyn TheTrait>.

Is this what you meant? https://github.com/mozilla/neqo/pull/2592

I also wrote a prototype of how it should work in neqo_glue where we implement a trait: https://phabricator.services.mozilla.com/D245906 :) The phab patch will absolutely change, the intent was to check if we're on the same page about how it all should work (obviously, third_parts/rust will be gone)

Frosne avatar Apr 17 '25 14:04 Frosne

Are we waiting on anything for this PR? Code changes by @Frosne or an updated review by @martinthomson?

larseggert avatar May 22 '25 05:05 larseggert

I don't think that this is the one we want. #2592 seems better, but I'm still waiting on an update there.

martinthomson avatar May 22 '25 05:05 martinthomson

OK. @Frosne, if this PR is OBE, please close?

larseggert avatar May 22 '25 06:05 larseggert