rdedup icon indicating copy to clipboard operation
rdedup copied to clipboard

speed benchmark

Open legrostdg opened this issue 9 years ago • 8 comments

It would help to have a speed benchmark.

legrostdg avatar Nov 30 '16 23:11 legrostdg

Can you explain further?

phillipCouto avatar Dec 16 '16 12:12 phillipCouto

The benchmark would compare first/consequent savings, restore, etc. on local and remote host for several software: rdedup, attic, borg, other?

legrostdg avatar Dec 16 '16 12:12 legrostdg

So a benchmark of performance against other projects?

phillipCouto avatar Dec 16 '16 12:12 phillipCouto

yes!

legrostdg avatar Dec 16 '16 12:12 legrostdg

@dpc what do you think?

I think we can create a benchmark suite to track performance regression with development but does benchmarking rdedup against other projects make sense to include in the project or maybe just a gist in github with a blurp of the results in the readme?

phillipCouto avatar Dec 16 '16 12:12 phillipCouto

Would definitely be great to have an internal benchmark and it would be useful for ongoing development with something like cargo-cmpbench in .travis.yml

The external comparison benchmark would be neat as well: to get some ballpark how do rdedup compare. However, given that rdedup using public key encryption and AFAIK other ones are using symmetric encryption it's nothing to stress about.

With both issues - PRs are always welcome. :)

dpc avatar Dec 16 '16 18:12 dpc

Some WIP: https://github.com/gilbertchen/benchmarking/pull/3

It looks that rdedup is doing really well. On my machine, it seems faster/as fast as the best tool in each categroy: both backup and restore operations. The repository size ends up to be 1.4G with 1M chunks and 1.2G with smaller chunks. Which is OKish, but could be improved.

dpc avatar Jul 19 '17 06:07 dpc

With xz2 compression repo size goes down to 1007M, but the backup times skyrocket. bzip2 compression results in 1.1G with times just twice as long as default deflate.

dpc avatar Jul 19 '17 07:07 dpc