csvdiff icon indicating copy to clipboard operation
csvdiff copied to clipboard

Published 20 hours ago • verified publisher icon aswinkarthik

Big File compare slowly

Open xidianwlc opened this issue 4 years ago • 1 comments

when a file is 1TB then csvdiff was slow and cpu used too much

you can use binary diff algorighm

if xxHash(src multilines) == xxHash(dst multilines) then continue else compare line by line

xidianwlc avatar Mar 19 '20 11:03 xidianwlc

Will you be able to share some stats about this:

  1. How big was base and delta file?
  2. Did it finish? If so, how much time did it take?

aswinkarthik avatar Mar 20 '20 09:03 aswinkarthik