Paweł Redzyński

s3 benchmarks: recovering from errors

4

s3 benchmarks are heavily influenced with aws and network moods For example latest @isidentical [change](https://github.com/iterative/dvc-bench/pull/258) had to be rerun until the PR build starts passing. We need to investigate if...

push/import: reconsider benchmarks using real data

1

Currently in cases where we want to use "real" data we use `cats_dogs` dataset which is 25k images, altogether ~800mb. Benchmarks using them, tend to take a lot of time....

Example: It seems that as of `1.11.0` dvc version we are downloading output upon `api.get_url`. Some benchmark could detect it.

It is possible to specify the whole directory as a plot. It would be good to be able to granularly specify config parameters for particular files in the directory.