uv
uv copied to clipboard
Add airflow benchmark
Summary
This seems to be one of the most consistent benchmark cases we have in terms of standard deviation:
➜ hyperfine "target/profiling/main pip compile scripts/requirements/airflow.in" --runs 200
Benchmark 1: target/profiling/main pip compile scripts/requirements/airflow.in
Time (mean ± σ): 292.6 ms ± 6.6 ms [User: 414.1 ms, System: 194.2 ms]
Range (min … max): 282.7 ms … 320.1 ms 200 runs
For smaller benchmarks, scispacy and dtlssocket seem to be a bit more consistent than our current jupyter benchmark, but it hasn't given us any problems so I'll leave it for now.
Hm we might have to install source distributions in a separate CI step to get CodSpeed to work.
Since this is a cached case benchmark, we can build all dependencies (pip build should do), upload them to github pages and use --find-links. I can't tell if the effort is justified here though.
In https://github.com/astral-sh/uv/pull/3688 we had a massive speedup on airflow but nearly no effect on jupyter, so this would be helpful for this case.
CodSpeed Performance Report
Merging #3643 will not alter performance
Comparing ibraheemdev:airflow-bench (bacd1a0) with main (8f0f4d2)
Summary
✅ 12 untouched benchmarks
🆕 1 new benchmarks
Benchmarks breakdown
| Benchmark | main |
ibraheemdev:airflow-bench |
Change | |
|---|---|---|---|---|
| 🆕 | resolve_warm_airflow |
N/A | 1.5 s | N/A |