websync icon indicating copy to clipboard operation
websync copied to clipboard

compare md5 on sync

Open phils opened this issue 7 years ago • 2 comments

An oft requested feature for awscli is to compare more than just size or date when doing the sync. I understand that the md5 isn't returned when you do a standard list AWS call, so it is not efficient to do it this way.

However, s3cmd has a solution where it stores the md5 in the object metadata when transferring, so that on a subsequent sync, that metadata is returned and it can be used to compare. s3cmd however has other issues such as not supporting instance metadata or standard API key locations for auth which makes it unattractive.

If websync implemented the same md5 metadata strategy it would be a nice addition.

phils avatar Feb 03 '18 00:02 phils

Thanks for the feature request, and sorry for the late reply! Love the idea, it seems like it would either require that one has configured websync (let's assume we support this) to store the md5 hash as a metadata attribute, or that websync does this by default. Do I have that right? Either way, I don't think it's too much of an issue. I am wondering, what your thoughts would be on whether the md5 hash is something done by default, or as option.

nickpisacane avatar Mar 27 '18 02:03 nickpisacane

I guess, as a fall back, we could just generate the md5 for a given file, if the given metadata attribute is missing.

nickpisacane avatar Mar 27 '18 02:03 nickpisacane