benchmarking
benchmarking copied to clipboard
Oficial rules?
So I have been using this benchmark to optimize rdedup
, and I was doing some test and improvements, etc. At some point, I'd like to add rdedup
to the official "results". And I am wondering what are the actual rules? Eg. 1MB chunks are big, and rdedup
results are much better with smaller chunks. I am guessing other tools might also benefit from different settings too.
As every benchmark encourages competition, it is only fair if the rules are clear.
For starter:
- can we tune settings for every program?
- do have to stick to the same settings in "linux" (small files) vs "vm" (big image)?
There aren't such rules and I don't think there should be. All you need to do is to state your settings clearly to the degree that other people can reproduce your results easily. In my option this benchmark serves two purposes: to give users a rough idea of how fast each tool is, and to help developers identify if there is any room for improvement.
As for the size of chunks, chunks smaller than 1MB would incur too much overhead when uploading to cloud storages.