feat: Use `jemalloc` as allocator
Gecko uses (it's own flavor of) jemalloc, including for Rust. The intent here is to get the standalone demo client and server, and the benchmarks, closer to Gecko.
As discussed with @valenting.
TODO: Is there a way to put this behind a feature or something, so that the tikv-jemallocator crate doesn't become part of the Gecko deps when vendored in?
Failed Interop Tests
QUIC Interop Runner, client vs. server, differences relative to 3a429ef8b795dc54dfa59c4edb435943bcf59439.
neqo-latest as client
- neqo-latest vs. go-x-net: BP BA
- neqo-latest vs. haproxy: :rocket:~~L1~~ BP BA
- neqo-latest vs. kwik: C1 BP BA
- neqo-latest vs. linuxquic: L1 C1
- neqo-latest vs. lsquic: E L1 C1
- neqo-latest vs. msquic: :rocket:~~R Z~~ A L1 C1
- neqo-latest vs. mvfst: A L1 C1
- neqo-latest vs. neqo: A
- neqo-latest vs. neqo-latest: A
- neqo-latest vs. nginx: BP BA
- neqo-latest vs. ngtcp2: E CM
- neqo-latest vs. picoquic: :warning:Z E A L1 :rocket:~~C1~~
- neqo-latest vs. quic-go: A
- neqo-latest vs. quiche: :warning:L1 BP BA
- neqo-latest vs. quinn: A
- neqo-latest vs. s2n-quic: E BA CM
- neqo-latest vs. tquic: S :warning:A BP BA
- neqo-latest vs. xquic: H DC LR C20 M R Z 3 B U A L1 L2 C1 C2 6 BP BA
neqo-latest as server
- aioquic vs. neqo-latest: :rocket:~~C1~~ CM
- go-x-net vs. neqo-latest: CM
- kwik vs. neqo-latest: BP BA CM
- lsquic vs. neqo-latest: :warning:BA
- msquic vs. neqo-latest: U :warning:L1 BA CM
- mvfst vs. neqo-latest: Z A L1 C1 CM
- neqo vs. neqo-latest: A
- openssl vs. neqo-latest: LR M A CM
- quic-go vs. neqo-latest: :warning:BA CM
- quiche vs. neqo-latest: :rocket:~~C1~~ CM
- quinn vs. neqo-latest: :rocket:~~L1 C1~~ V2 CM
- s2n-quic vs. neqo-latest: CM
- tquic vs. neqo-latest: CM
- xquic vs. neqo-latest: M CM
All results
Succeeded Interop Tests
QUIC Interop Runner, client vs. server
neqo-latest as client
- neqo-latest vs. aioquic: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2 BP BA
- neqo-latest vs. go-x-net: H DC LR M B U A L2 C2 6
- neqo-latest vs. haproxy: H DC LR C20 M S R Z 3 B U A :rocket:~~L1~~ L2 C1 C2 6 V2
- neqo-latest vs. kwik: H DC LR C20 M S R Z 3 B U A L1 L2 C2 6 V2
- neqo-latest vs. linuxquic: H DC LR C20 M S R Z 3 B U E A L2 C2 6 V2 BP BA CM
- neqo-latest vs. lsquic: H DC LR C20 M S R Z 3 B U A L2 C2 6 V2 BP BA CM
- neqo-latest vs. msquic: H DC LR C20 M S :rocket:~~R Z~~ B U L2 C2 6 V2 BP BA
- neqo-latest vs. mvfst: H DC LR M R Z 3 B U L2 C2 6 BP BA
- neqo-latest vs. neqo: H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM
- neqo-latest vs. neqo-latest: H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM
- neqo-latest vs. nginx: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6
- neqo-latest vs. ngtcp2: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2 BP BA
- neqo-latest vs. picoquic: H DC LR C20 M S R :warning:Z 3 B U L2 :rocket:~~C1~~ C2 6 V2 BP BA
- neqo-latest vs. quic-go: H DC LR C20 M S R Z 3 B U L1 L2 C1 C2 6 BP BA
- neqo-latest vs. quiche: H DC LR C20 M S R Z 3 B U A :warning:L1 L2 C1 C2 6
- neqo-latest vs. quinn: H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 BP BA
- neqo-latest vs. s2n-quic: H DC LR C20 M S R 3 B U A L1 L2 C1 C2 6 BP
- neqo-latest vs. tquic: H DC LR C20 M R Z 3 B U :warning:A L1 L2 C1 C2 6
neqo-latest as server
- aioquic vs. neqo-latest: H DC LR C20 M S R Z 3 B A L1 L2 :rocket:~~C1~~ C2 6 V2 BP BA
- chrome vs. neqo-latest: 3
- go-x-net vs. neqo-latest: H DC LR M B U A L2 C2 6 BP BA
- kwik vs. neqo-latest: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2
- linuxquic vs. neqo-latest: H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM
- lsquic vs. neqo-latest: H DC LR C20 M S R 3 B E A L1 L2 C1 C2 6 V2 BP :warning:BA CM
- msquic vs. neqo-latest: H DC LR C20 M S R Z B A :warning:L1 L2 C1 C2 6 V2 BP :warning:BA
- mvfst vs. neqo-latest: H DC LR M 3 B L2 C2 6 BP BA
- neqo vs. neqo-latest: H DC LR C20 M S R Z 3 B U E L1 L2 C1 C2 6 V2 BP BA CM
- ngtcp2 vs. neqo-latest: H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM
- openssl vs. neqo-latest: H DC C20 S R 3 B L2 C2 6 BP BA
- picoquic vs. neqo-latest: H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2 BP BA CM
- quic-go vs. neqo-latest: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 BP :warning:BA
- quiche vs. neqo-latest: H DC LR M S R Z 3 B A L1 L2 :rocket:~~C1~~ C2 6 BP BA
- quinn vs. neqo-latest: H DC LR C20 M S R Z 3 B U E A :rocket:~~L1~~ L2 :rocket:~~C1~~ C2 6 BP BA
- s2n-quic vs. neqo-latest: H DC LR M S R 3 B E A L1 L2 C1 C2 6 BP BA
- tquic vs. neqo-latest: H DC LR M S R Z 3 B A L1 L2 C1 C2 6 BP BA
- xquic vs. neqo-latest: H DC LR C20 S R Z 3 B U A L1 L2 C1 C2 6 BP BA
Unsupported Interop Tests
QUIC Interop Runner, client vs. server
neqo-latest as client
- neqo-latest vs. aioquic: E CM
- neqo-latest vs. go-x-net: C20 S R Z 3 E L1 C1 V2 CM
- neqo-latest vs. haproxy: E CM
- neqo-latest vs. kwik: E CM
- neqo-latest vs. msquic: 3 E CM
- neqo-latest vs. mvfst: C20 S E V2 CM
- neqo-latest vs. nginx: E V2 CM
- neqo-latest vs. picoquic: CM
- neqo-latest vs. quic-go: E V2 CM
- neqo-latest vs. quiche: E V2 CM
- neqo-latest vs. quinn: V2 CM
- neqo-latest vs. s2n-quic: Z V2
- neqo-latest vs. tquic: E V2 CM
- neqo-latest vs. xquic: S E V2 CM
neqo-latest as server
- aioquic vs. neqo-latest: U E
- chrome vs. neqo-latest: H DC LR C20 M S R Z B U E A L1 L2 C1 C2 6 V2 BP BA CM
- go-x-net vs. neqo-latest: C20 S R Z 3 E L1 C1 V2
- kwik vs. neqo-latest: E
- lsquic vs. neqo-latest: Z U
- msquic vs. neqo-latest: 3 E
- mvfst vs. neqo-latest: C20 S R U E V2
- openssl vs. neqo-latest: Z U E L1 C1 V2
- quic-go vs. neqo-latest: E V2
- quiche vs. neqo-latest: C20 U E V2
- s2n-quic vs. neqo-latest: C20 Z U V2
- tquic vs. neqo-latest: C20 U E V2
- xquic vs. neqo-latest: E V2
Client/server transfer results
Performance differences relative to 3a429ef8b795dc54dfa59c4edb435943bcf59439.
Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.
| Client vs. server (params) | Mean ± σ | Min | Max | MiB/s ± σ | Δ main |
Δ main |
|---|---|---|---|---|---|---|
| google vs. google | 454.9 ± 3.5 | 448.7 | 465.6 | 70.3 ± 9.1 | ||
| google vs. neqo (cubic, paced) | 284.5 ± 4.5 | 277.7 | 300.3 | 112.5 ± 7.1 | :broken_heart: 7.5 | 2.7% |
| msquic vs. msquic | 159.2 ± 17.5 | 137.5 | 232.7 | 200.9 ± 1.8 | ||
| msquic vs. neqo (cubic, paced) | 186.1 ± 23.9 | 161.8 | 342.1 | 171.9 ± 1.3 | 4.2 | 2.3% |
| neqo vs. google (cubic, paced) | 758.4 ± 4.2 | 751.3 | 770.9 | 42.2 ± 7.6 | 0.8 | 0.1% |
| neqo vs. msquic (cubic, paced) | 157.1 ± 4.1 | 149.4 | 165.0 | 203.6 ± 7.8 | -0.9 | -0.6% |
| neqo vs. neqo (cubic) | 97.7 ± 8.5 | 81.3 | 112.8 | 327.5 ± 3.8 | :broken_heart: 9.4 | 10.7% |
| neqo vs. neqo (cubic, paced) | 96.3 ± 7.4 | 85.0 | 113.6 | 332.3 ± 4.3 | :broken_heart: 5.8 | 6.4% |
| neqo vs. neqo (reno) | 94.1 ± 7.1 | 83.8 | 111.0 | 340.0 ± 4.5 | :broken_heart: 4.8 | 5.4% |
| neqo vs. neqo (reno, paced) | 94.4 ± 7.3 | 82.5 | 112.6 | 338.9 ± 4.4 | :broken_heart: 4.8 | 5.4% |
| neqo vs. quiche (cubic, paced) | 194.0 ± 5.0 | 187.0 | 208.7 | 165.0 ± 6.4 | :green_heart: -2.5 | -1.3% |
| neqo vs. s2n (cubic, paced) | 222.6 ± 4.9 | 213.2 | 234.5 | 143.8 ± 6.5 | :broken_heart: 4.0 | 1.8% |
| quiche vs. neqo (cubic, paced) | 152.0 ± 4.4 | 139.8 | 163.3 | 210.5 ± 7.3 | :green_heart: -5.8 | -3.7% |
| quiche vs. quiche | 146.1 ± 5.5 | 136.9 | 157.0 | 219.1 ± 5.8 | ||
| s2n vs. neqo (cubic, paced) | 176.2 ± 4.6 | 167.2 | 186.6 | 181.6 ± 7.0 | -0.4 | -0.2% |
| s2n vs. s2n | 251.8 ± 29.8 | 232.3 | 367.7 | 127.1 ± 1.1 |
Download data for profiler.firefox.com or download performance comparison data.
Bencher Report
| Branch | feat-jemalloc |
| Testbed | On-prem |
🚨 3 Alerts
| Iteration | Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|---|
| 0 | google vs. neqo (cubic, paced) | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 284.45 ms(+2.44%)Baseline: 277.68 ms | 280.62 ms (101.36%) |
| 4 | neqo vs. neqo (cubic) | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 97.72 ms(+7.31%)Baseline: 91.06 ms | 94.99 ms (102.88%) |
| 5 | neqo vs. neqo (cubic, paced) | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 96.30 ms(+4.28%)Baseline: 92.35 ms | 95.82 ms (100.50%) |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| google vs. neqo (cubic, paced) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 284.45 ms(+2.44%)Baseline: 277.68 ms | 280.62 ms (101.36%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| msquic vs. neqo (cubic, paced) | 📈 view plot 🚷 view threshold | 186.13 ms(-4.49%)Baseline: 194.88 ms | 229.72 ms (81.02%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. google (cubic, paced) | 📈 view plot 🚷 view threshold | 758.42 ms(+0.09%)Baseline: 757.73 ms | 764.82 ms (99.16%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. msquic (cubic, paced) | 📈 view plot 🚷 view threshold | 157.14 ms(+0.12%)Baseline: 156.95 ms | 158.90 ms (98.89%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. neqo (cubic) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 97.72 ms(+7.31%)Baseline: 91.06 ms | 94.99 ms (102.88%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. neqo (cubic, paced) | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 96.30 ms(+4.28%)Baseline: 92.35 ms | 95.82 ms (100.50%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. neqo (reno) | 📈 view plot 🚷 view threshold | 94.13 ms(+3.46%)Baseline: 90.98 ms | 94.28 ms (99.84%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. neqo (reno, paced) | 📈 view plot 🚷 view threshold | 94.43 ms(+2.31%)Baseline: 92.29 ms | 95.51 ms (98.87%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. quiche (cubic, paced) | 📈 view plot 🚷 view threshold | 193.98 ms(+0.04%)Baseline: 193.90 ms | 197.32 ms (98.31%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| neqo vs. s2n (cubic, paced) | 📈 view plot 🚷 view threshold | 222.57 ms(+0.75%)Baseline: 220.90 ms | 223.65 ms (99.51%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| quiche vs. neqo (cubic, paced) | 📈 view plot 🚷 view threshold | 152.04 ms(-0.01%)Baseline: 152.06 ms | 157.75 ms (96.38%) |
| Benchmark | Latency | Benchmark Result milliseconds (ms) (Result Δ%) | Upper Boundary milliseconds (ms) (Limit %) |
|---|---|---|---|
| s2n vs. neqo (cubic, paced) | 📈 view plot 🚷 view threshold | 176.18 ms(+1.41%)Baseline: 173.73 ms | 177.87 ms (99.05%) |
Benchmark results
Performance differences relative to 3a429ef8b795dc54dfa59c4edb435943bcf59439.
1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: :broken_heart: Performance has regressed.
time: [210.28 ms 210.62 ms 210.94 ms]
thrpt: [474.06 MiB/s 474.80 MiB/s 475.55 MiB/s]
change:
time: [+3.6042% +3.8638% +4.1170%] (p = 0.00 −3.7201% −3.4788%]
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low severe
3 (3.00%) high mild
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: :green_heart: Performance has improved.
time: [248.23 ms 249.98 ms 251.77 ms]
thrpt: [39.719 Kelem/s 40.003 Kelem/s 40.285 Kelem/s]
change:
time: [−17.201% −16.487% −15.734%] (p = 0.00 +19.741% +20.774%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: Change within noise threshold.
time: [28.774 ms 28.901 ms 29.046 ms]
thrpt: [34.429 B/s 34.601 B/s 34.754 B/s]
change:
time: [+0.7745% +1.3364% +1.9905%] (p = 0.00 −1.3188% −0.7686%]
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: :broken_heart: Performance has regressed.
time: [251.59 ms 259.98 ms 268.20 ms]
thrpt: [372.86 MiB/s 384.65 MiB/s 397.47 MiB/s]
change:
time: [+20.908% +25.003% +29.247%] (p = 0.00 −20.002% −17.292%]
decode 4096 bytes, mask ff: No change in performance detected.
time: [11.585 µs 11.635 µs 11.692 µs]
change: [−0.6360% −0.2505% +0.1056%] (p = 0.22 > 0.05)
Found 22 outliers among 100 measurements (22.00%)
5 (5.00%) low severe
6 (6.00%) low mild
11 (11.00%) high severe
decode 1048576 bytes, mask ff: No change in performance detected.
time: [3.0159 ms 3.0257 ms 3.0369 ms]
change: [−0.6545% −0.1304% +0.3521%] (p = 0.62 > 0.05)
Found 12 outliers among 100 measurements (12.00%)
2 (2.00%) high mild
10 (10.00%) high severe
decode 4096 bytes, mask 7f: :green_heart: Performance has improved.
time: [18.280 µs 18.329 µs 18.383 µs]
change: [−9.5837% −8.8435% −8.2866%] (p = 0.00 Found 15 outliers among 100 measurements (15.00%)
2 (2.00%) low severe
1 (1.00%) low mild
12 (12.00%) high severedecode 1048576 bytes, mask 7f: Change within noise threshold.
time: [4.9860 ms 4.9972 ms 5.0100 ms]
change: [−1.5365% −1.1416% −0.7566%] (p = 0.00 Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) high mild
12 (12.00%) high severedecode 4096 bytes, mask 3f: :green_heart: Performance has improved.
time: [7.3546 µs 7.4885 µs 7.6248 µs]
change: [−10.560% −9.3678% −8.4289%] (p = 0.00 decode 1048576 bytes, mask 3f: No change in performance detected.
time: [1.5867 ms 1.5959 ms 1.6087 ms]
change: [−0.5228% +0.2176% +1.0189%] (p = 0.64 > 0.05)
Found 6 outliers among 100 measurements (6.00%)
6 (6.00%) high severe
1-streams/each-1000-bytes/wallclock-time: :green_heart: Performance has improved.
time: [571.70 µs 573.68 µs 575.93 µs]
change: [−3.7551% −3.2092% −2.6778%] (p = 0.00 Found 7 outliers among 100 measurements (7.00%)
7 (7.00%) high severe
1-streams/each-1000-bytes/simulated-time
time: [118.87 ms 119.08 ms 119.27 ms]
thrpt: [8.1878 KiB/s 8.2012 KiB/s 8.2152 KiB/s]
change:
time: [−0.2181% +0.0382% +0.2976%] (p = 0.77 > 0.05)
thrpt: [−0.2967% −0.0382% +0.2186%]
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild1000-streams/each-1-bytes/wallclock-time: :green_heart: Performance has improved.
time: [11.283 ms 11.299 ms 11.314 ms]
change: [−19.471% −19.287% −19.107%] (p = 0.00 1000-streams/each-1-bytes/simulated-time: No change in performance detected.
time: [14.990 s 15.003 s 15.016 s]
thrpt: [66.594 B/s 66.651 B/s 66.710 B/s]
change:
time: [−0.0725% +0.0555% +0.1845%] (p = 0.40 > 0.05)
thrpt: [−0.1841% −0.0555% +0.0725%]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
1000-streams/each-1000-bytes/wallclock-time: :green_heart: Performance has improved.
time: [46.077 ms 46.231 ms 46.389 ms]
change: [−9.0378% −8.4025% −7.8606%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
1000-streams/each-1000-bytes/simulated-time
time: [18.637 s 18.785 s 18.932 s]
thrpt: [51.581 KiB/s 51.987 KiB/s 52.398 KiB/s]
change:
time: [−1.9572% −0.6863% +0.6017%] (p = 0.30 > 0.05)
thrpt: [−0.5981% +0.6910% +1.9963%]
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mildcoalesce_acked_from_zero 1+1 entries: No change in performance detected.
time: [89.865 ns 90.372 ns 90.818 ns]
change: [−1.1626% +0.2747% +1.2675%] (p = 0.72 > 0.05)
coalesce_acked_from_zero 3+1 entries: No change in performance detected.
time: [107.64 ns 108.26 ns 108.88 ns]
change: [−0.5086% +0.4467% +1.2625%] (p = 0.34 > 0.05)
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
coalesce_acked_from_zero 10+1 entries: No change in performance detected.
time: [105.65 ns 105.99 ns 106.43 ns]
change: [−0.0663% +0.6396% +1.7367%] (p = 0.16 > 0.05)
Found 18 outliers among 100 measurements (18.00%)
5 (5.00%) low severe
3 (3.00%) low mild
2 (2.00%) high mild
8 (8.00%) high severe
coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
time: [89.357 ns 89.512 ns 89.680 ns]
change: [−0.9987% +0.0224% +0.9031%] (p = 0.96 > 0.05)
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
RxStreamOrderer::inbound_frame(): :green_heart: Performance has improved.
time: [31.264 ms 31.349 ms 31.462 ms]
change: [−71.916% −71.812% −71.707%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severesent::Packets::take_ranges: :broken_heart: Performance has regressed.
time: [6.6977 µs 7.0407 µs 7.3534 µs]
change: [+34.885% +43.517% +53.241%] (p = 0.00 Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severetransfer/pacing-false/varying-seeds/wallclock-time/run: :green_heart: Performance has improved.
time: [24.614 ms 24.662 ms 24.722 ms]
change: [−7.7771% −7.5115% −7.2411%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severetransfer/pacing-false/varying-seeds/simulated-time/run: Change within noise threshold.
time: [25.110 s 25.142 s 25.174 s]
thrpt: [162.71 KiB/s 162.91 KiB/s 163.12 KiB/s]
change:
time: [−0.3934% −0.1974% −0.0025%] (p = 0.05 +0.1978% +0.3950%]
transfer/pacing-true/varying-seeds/wallclock-time/run: :green_heart: Performance has improved.
time: [25.221 ms 25.280 ms 25.342 ms]
change: [−7.8184% −7.4803% −7.1217%] (p = 0.00 Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
4 (4.00%) high mildtransfer/pacing-true/varying-seeds/simulated-time/run: No change in performance detected.
time: [24.949 s 24.992 s 25.036 s]
thrpt: [163.60 KiB/s 163.89 KiB/s 164.17 KiB/s]
change:
time: [−0.1926% +0.0513% +0.3077%] (p = 0.68 > 0.05)
thrpt: [−0.3068% −0.0512% +0.1929%]
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high mild
transfer/pacing-false/same-seed/wallclock-time/run: :green_heart: Performance has improved.
time: [24.320 ms 24.355 ms 24.406 ms]
change: [−6.8920% −6.7405% −6.5443%] (p = 0.00 Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severetransfer/pacing-false/same-seed/simulated-time/run: No change in performance detected.
time: [25.152 s 25.152 s 25.152 s]
thrpt: [162.85 KiB/s 162.85 KiB/s 162.85 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000%] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000%]
transfer/pacing-true/same-seed/wallclock-time/run: :green_heart: Performance has improved.
time: [25.547 ms 25.571 ms 25.598 ms]
change: [−7.6366% −7.4521% −7.2947%] (p = 0.00 Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severetransfer/pacing-true/same-seed/simulated-time/run: No change in performance detected.
time: [25.588 s 25.588 s 25.588 s]
thrpt: [160.07 KiB/s 160.07 KiB/s 160.07 KiB/s]
change:
time: [+0.0000% +0.0000% +0.0000%] (p = NaN > 0.05)
thrpt: [+0.0000% +0.0000% +0.0000%]
Download data for profiler.firefox.com or download performance comparison data.
Bencher Report
| Branch | feat-jemalloc |
| Testbed | On-prem |
🚨 5 Alerts
| Benchmark | Measure Units | View | Benchmark Result (Result Δ%) | Upper Boundary (Limit %) |
|---|---|---|---|---|
| 1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 259.98 ms(+23.52%)Baseline: 210.47 ms | 223.03 ms (116.57%) |
| 1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client | Latency milliseconds (ms) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 28.90 ms(+1.81%)Baseline: 28.39 ms | 28.89 ms (100.03%) |
| coalesce_acked_from_zero 1+1 entries | Latency nanoseconds (ns) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 90.37 ns(+1.92%)Baseline: 88.67 ns | 89.43 ns (101.06%) |
| coalesce_acked_from_zero 3+1 entries | Latency nanoseconds (ns) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 108.26 ns(+1.48%)Baseline: 106.68 ns | 107.69 ns (100.53%) |
| sent::Packets::take_ranges | Latency microseconds (µs) | 📈 plot 🚷 threshold 🚨 alert (🔔) | 7.04 µs(+47.32%)Baseline: 4.78 µs | 5.30 µs (132.84%) |
Click to view all benchmark results
| Benchmark | Latency | Benchmark Result nanoseconds (ns) (Result Δ%) | Upper Boundary nanoseconds (ns) (Limit %) |
|---|---|---|---|
| 1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 259,980,000.00 ns(+23.52%)Baseline: 210,472,962.96 ns | 223,033,276.58 ns (116.57%) |
| 1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client | 📈 view plot 🚷 view threshold | 210,620,000.00 ns(+3.08%)Baseline: 204,331,481.48 ns | 212,941,037.21 ns (98.91%) |
| 1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 28,901,000.00 ns(+1.81%)Baseline: 28,387,081.48 ns | 28,892,398.05 ns (100.03%) |
| 1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client | 📈 view plot 🚷 view threshold | 249,980,000.00 ns(-15.38%)Baseline: 295,400,222.22 ns | 308,509,573.65 ns (81.03%) |
| 1-streams/each-1000-bytes/simulated-time | 📈 view plot 🚷 view threshold | 119,080,000.00 ns(+0.84%)Baseline: 118,084,518.52 ns | 120,820,164.70 ns (98.56%) |
| 1-streams/each-1000-bytes/wallclock-time | 📈 view plot 🚷 view threshold | 573,680.00 ns(-4.56%)Baseline: 601,062.00 ns | 625,796.72 ns (91.67%) |
| 1000-streams/each-1-bytes/simulated-time | 📈 view plot 🚷 view threshold | 15,003,000,000.00 ns(+0.08%)Baseline: 14,990,948,148.15 ns | 15,008,937,359.73 ns (99.96%) |
| 1000-streams/each-1-bytes/wallclock-time | 📈 view plot 🚷 view threshold | 11,299,000.00 ns(-21.08%)Baseline: 14,316,851.85 ns | 15,191,509.58 ns (74.38%) |
| 1000-streams/each-1000-bytes/simulated-time | 📈 view plot 🚷 view threshold | 18,785,000,000.00 ns(-0.55%)Baseline: 18,888,051,851.85 ns | 19,097,711,273.73 ns (98.36%) |
| 1000-streams/each-1000-bytes/wallclock-time | 📈 view plot 🚷 view threshold | 46,231,000.00 ns(-12.89%)Baseline: 53,069,940.74 ns | 59,442,332.22 ns (77.77%) |
| RxStreamOrderer::inbound_frame() | 📈 view plot 🚷 view threshold | 31,349,000.00 ns(-71.33%)Baseline: 109,349,103.70 ns | 125,359,644.74 ns (25.01%) |
| coalesce_acked_from_zero 1+1 entries | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 90.37 ns(+1.92%)Baseline: 88.67 ns | 89.43 ns (101.06%) |
| coalesce_acked_from_zero 10+1 entries | 📈 view plot 🚷 view threshold | 105.99 ns(-0.16%)Baseline: 106.16 ns | 107.13 ns (98.93%) |
| coalesce_acked_from_zero 1000+1 entries | 📈 view plot 🚷 view threshold | 89.51 ns(-0.39%)Baseline: 89.86 ns | 94.61 ns (94.61%) |
| coalesce_acked_from_zero 3+1 entries | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 108.26 ns(+1.48%)Baseline: 106.68 ns | 107.69 ns (100.53%) |
| decode 1048576 bytes, mask 3f | 📈 view plot 🚷 view threshold | 1,595,900.00 ns(+0.21%)Baseline: 1,592,600.74 ns | 1,599,782.97 ns (99.76%) |
| decode 1048576 bytes, mask 7f | 📈 view plot 🚷 view threshold | 4,997,200.00 ns(-1.18%)Baseline: 5,056,740.00 ns | 5,079,455.13 ns (98.38%) |
| decode 1048576 bytes, mask ff | 📈 view plot 🚷 view threshold | 3,025,700.00 ns(-0.19%)Baseline: 3,031,547.41 ns | 3,043,901.48 ns (99.40%) |
| decode 4096 bytes, mask 3f | 📈 view plot 🚷 view threshold | 7,488.50 ns(-9.67%)Baseline: 8,289.85 ns | 8,459.06 ns (88.53%) |
| decode 4096 bytes, mask 7f | 📈 view plot 🚷 view threshold | 18,329.00 ns(-8.33%)Baseline: 19,994.61 ns | 20,343.46 ns (90.10%) |
| decode 4096 bytes, mask ff | 📈 view plot 🚷 view threshold | 11,635.00 ns(-1.06%)Baseline: 11,759.86 ns | 12,005.00 ns (96.92%) |
| sent::Packets::take_ranges | 📈 view plot 🚷 view threshold 🚨 view alert (🔔) | 7,040.70 ns(+47.32%)Baseline: 4,779.09 ns | 5,300.19 ns (132.84%) |
| transfer/pacing-false/same-seed/simulated-time/run | 📈 view plot 🚷 view threshold | 25,152,000,000.00 ns | |
| transfer/pacing-false/same-seed/wallclock-time/run | 📈 view plot 🚷 view threshold | 24,355,000.00 ns(-6.73%)Baseline: 26,113,210.53 ns | 27,032,527.91 ns (90.10%) |
| transfer/pacing-false/varying-seeds/simulated-time/run | 📈 view plot 🚷 view threshold | 25,142,000,000.00 ns(-0.09%)Baseline: 25,164,135,338.35 ns | 25,207,302,667.08 ns (99.74%) |
| transfer/pacing-false/varying-seeds/wallclock-time/run | 📈 view plot 🚷 view threshold | 24,662,000.00 ns(-6.77%)Baseline: 26,452,857.14 ns | 27,317,555.81 ns (90.28%) |
| transfer/pacing-true/same-seed/simulated-time/run | 📈 view plot 🚷 view threshold | 25,588,000,000.00 ns | |
| transfer/pacing-true/same-seed/wallclock-time/run | 📈 view plot 🚷 view threshold | 25,571,000.00 ns(-7.42%)Baseline: 27,621,473.68 ns | 28,598,923.46 ns (89.41%) |
| transfer/pacing-true/varying-seeds/simulated-time/run | 📈 view plot 🚷 view threshold | 24,992,000,000.00 ns(-0.01%)Baseline: 24,994,075,187.97 ns | 25,043,826,395.73 ns (99.79%) |
| transfer/pacing-true/varying-seeds/wallclock-time/run | 📈 view plot 🚷 view threshold | 25,280,000.00 ns(-6.35%)Baseline: 26,994,533.83 ns | 27,844,264.93 ns (90.79%) |
Hm. Worse than the default allocator.