benchmarking icon indicating copy to clipboard operation
benchmarking copied to clipboard

Oficial rules?

Open dpc opened this issue 7 years ago • 1 comments

So I have been using this benchmark to optimize rdedup, and I was doing some test and improvements, etc. At some point, I'd like to add rdedup to the official "results". And I am wondering what are the actual rules? Eg. 1MB chunks are big, and rdedup results are much better with smaller chunks. I am guessing other tools might also benefit from different settings too.

As every benchmark encourages competition, it is only fair if the rules are clear.

For starter:

  • can we tune settings for every program?
  • do have to stick to the same settings in "linux" (small files) vs "vm" (big image)?

dpc avatar Jul 24 '17 05:07 dpc

There aren't such rules and I don't think there should be. All you need to do is to state your settings clearly to the degree that other people can reproduce your results easily. In my option this benchmark serves two purposes: to give users a rough idea of how fast each tool is, and to help developers identify if there is any room for improvement.

As for the size of chunks, chunks smaller than 1MB would incur too much overhead when uploading to cloud storages.

gilbertchen avatar Jul 24 '17 15:07 gilbertchen