dvc
dvc copied to clipboard
bench: add repro test
Closes https://github.com/iterative/dvc-bench/issues/468
Edit: the need for this is to help track performance for issues like https://github.com/iterative/dvc/issues/9085
Codecov Report
Attention: 8 lines in your changes are missing coverage. Please review.
Comparison is base (
f3fa674) 90.62% compared to head (d9abbe7) 90.60%. Report is 2 commits behind head on main.
| Files | Patch % | Lines |
|---|---|---|
| dvc/testing/benchmarks/cli/commands/test_repro.py | 0.00% | 8 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #10167 +/- ##
==========================================
- Coverage 90.62% 90.60% -0.02%
==========================================
Files 500 501 +1
Lines 37852 37879 +27
Branches 5503 5508 +5
==========================================
+ Hits 34305 34322 +17
- Misses 2908 2917 +9
- Partials 639 640 +1
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
It's going to be hard to write a fair benchmark for dvc repro.
I understand the motivation, but what you are testing here is dvc commit and that too is a no-op here because all of those were run already with run_copy() and is cached (mtimes, md5 hashes and the actual files).
Even if you made all of them --no-exec and tried to benchmark dvc repro, there is I/O happening in the subprocess cmd, which will be a significant part of the runtime.