opendal icon indicating copy to clipboard operation
opendal copied to clipboard

Add support for SHA1 checksum

Open prabirshrestha opened this issue 2 years ago • 2 comments

While opendal already supports md5, would it be possible to support SHA1 too? I'm working on building a webdav server that works with owncloud ios app and it apparently only support sha1 checksum even though you specify md5. I have created a feature request on their side too but given that some cloud providers support native sha1 checksum would be good to have it although I'm primarily looking for local file system support for sha1.

related: https://github.com/owncloud/ios-app/issues/1178

prabirshrestha avatar Jan 15 '23 21:01 prabirshrestha

Maybe related to https://github.com/datafuselabs/opendal/issues/157

For sha1 checksum support, do you mean calculate the sha1 checksum during upload(write)?

Xuanwo avatar Jan 16 '23 02:01 Xuanwo

Yes. Does seem similar to #157 but I want to also able to retrieve for existing ones.

I want to calculate during upload (write). There are two ways to do this. Use native sha1 provided by the backend or fallback to manual sha1 computation when backends doesn't support. S3 supports sha1 but others don't. Looking for something like this natively built in opendal. https://users.rust-lang.org/t/calculating-sha1-for-files-using-tokio-copy/87657/2. If the hash doesn't match after the upload (write) I want to it fail. OwnCloud iOS app sends OC-Checksum header with value SHA1:2b7bf5004c7b9bae41d0fbd1efe36b257cf34fb9 and since mobile phone networks cannot be reliable I want to make sure there is checksum validations so that uploads are not corrupted.

While I'm currently primarily looking for getting sha1 during upload I also want to get it by just reading the contents as I have cases where the user wants to reindex and the file already exists on the backend. In this case for backends that supports sha1 it would be fast but for others a full read to compute would be required. I also want to store this sha1 in the db so when viewing file details it can be shown.

prabirshrestha avatar Jan 16 '23 05:01 prabirshrestha