uv icon indicating copy to clipboard operation
uv copied to clipboard

Add airflow benchmark

Open ibraheemdev opened this issue 1 year ago • 3 comments

Summary

This seems to be one of the most consistent benchmark cases we have in terms of standard deviation:

➜  hyperfine "target/profiling/main pip compile scripts/requirements/airflow.in" --runs 200
Benchmark 1: target/profiling/main pip compile scripts/requirements/airflow.in
  Time (mean ± σ):     292.6 ms ±   6.6 ms    [User: 414.1 ms, System: 194.2 ms]
  Range (min … max):   282.7 ms … 320.1 ms    200 runs

For smaller benchmarks, scispacy and dtlssocket seem to be a bit more consistent than our current jupyter benchmark, but it hasn't given us any problems so I'll leave it for now.

ibraheemdev avatar May 17 '24 16:05 ibraheemdev

Hm we might have to install source distributions in a separate CI step to get CodSpeed to work.

ibraheemdev avatar May 17 '24 19:05 ibraheemdev

Since this is a cached case benchmark, we can build all dependencies (pip build should do), upload them to github pages and use --find-links. I can't tell if the effort is justified here though.

konstin avatar May 17 '24 19:05 konstin

In https://github.com/astral-sh/uv/pull/3688 we had a massive speedup on airflow but nearly no effect on jupyter, so this would be helpful for this case.

konstin avatar May 21 '24 12:05 konstin

CodSpeed Performance Report

Merging #3643 will not alter performance

Comparing ibraheemdev:airflow-bench (bacd1a0) with main (8f0f4d2)

Summary

✅ 12 untouched benchmarks

🆕 1 new benchmarks

Benchmarks breakdown

Benchmark main ibraheemdev:airflow-bench Change
🆕 resolve_warm_airflow N/A 1.5 s N/A

codspeed-hq[bot] avatar May 23 '24 16:05 codspeed-hq[bot]