matrix-rust-sdk icon indicating copy to clipboard operation
matrix-rust-sdk copied to clipboard

bench: Add the `room_list` bench

Open Hywan opened this issue 1 year ago • 1 comments

Example of the output:

room_list/sync/10       time:   [3.5673 ms 3.5899 ms 3.6181 ms]
                        thrpt:  [2.7639 Kelem/s 2.7856 Kelem/s 2.8032 Kelem/s]
Found 12 outliers among 100 measurements (12.00%)
  1 (1.00%) low severe
  3 (3.00%) high mild
  8 (8.00%) high severe
room_list/sync/100      time:   [3.6342 ms 3.6667 ms 3.7037 ms]
                        thrpt:  [27.000 Kelem/s 27.273 Kelem/s 27.516 Kelem/s]
Found 12 outliers among 100 measurements (12.00%)
  5 (5.00%) high mild
  7 (7.00%) high severe
room_list/sync/1000     time:   [30.426 ms 30.611 ms 30.810 ms]
                        thrpt:  [32.457 Kelem/s 32.668 Kelem/s 32.867 Kelem/s]
Found 20 outliers among 100 measurements (20.00%)
  13 (13.00%) high mild
  7 (7.00%) high severe
Benchmarking room_list/sync/10000: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 35.7s, or reduce sample count to 10.
room_list/sync/10000    time:   [353.56 ms 355.14 ms 356.97 ms]
                        thrpt:  [28.014 Kelem/s 28.158 Kelem/s 28.284 Kelem/s]
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe

Hywan avatar Jul 08 '24 10:07 Hywan

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.86%. Comparing base (8865e2f) to head (ebd0242). Report is 369 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3661      +/-   ##
==========================================
- Coverage   84.89%   84.86%   -0.03%     
==========================================
  Files         272      272              
  Lines       29167    29167              
==========================================
- Hits        24761    24753       -8     
- Misses       4406     4414       +8     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jul 08 '24 11:07 codecov[bot]

We're not particularly interested in this as of now and this is bitrotting, so closing now. If we ever decide to benchmark this properly let's reopen.

poljar avatar Dec 17 '24 13:12 poljar