Corruption with crypt backend when storing files on an SD Card (exFAT)

[dejl]

Hello

I am experimenting with using rclone crypt to securely store files on an external medium such as an SD card or USB flash drive, rather than using something like VeraCrypt.

I understand rclone is generally meant for cloud or remote storage providers and this works as intended just fine on the local disk.. with no issues.. however I have been experiencing intermittent but generally consistent "corruption" of the data that got written.. I can verify this by then unmounting, clearing vfs cache or switching to another machine and seeing decryption errors for files. I do not notice any errors at write time.

IO error: failed to authenticate decrypted block - bad password?

I don't experience this if I format the SD card as NTFS.

I have run h2testw on, and tested this with at least two different brand SD cards and adapters. My best guess is potentially an issue with using exFAT on the target? There is no file corruption with the same reader+sd card if I do not use rclone (copy files as normal), and several H2testw runs on the card results no error reading back.

I also ensure that the files written by rclone are completely flushed to disk and use the Safely Eject Tool in Windows to ensure that this is not the cause.

I have also tried to perform the writes using rclone copy rather than mount and still experience the same corruption. I have tried both Linux & Windows as the host and 1.65.2 & 1.65.1. I have tested both the cards with h2testw and both return no errors. If the remote is set to a local disk folder I experience no issues.

[local_crypt_ext]
type = crypt
remote = I:/Enc
password = <snip>
password2 = <snip>
filename_encryption = off
directory_name_encryption = false

where I:/ is an SD card formatted with exFAT.

Mounting with:

rclone mount local_crypt_ext:/ \\local\ext --vfs-cache-mode writes -v

I have also tried without the vfs-cache-mode flag and experience the same issue.

rclone v1.65.2
- os/version: Microsoft Windows 11 Education 22H2 (64 bit)
- os/kernel: 10.0.22621.3007 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.6
- go/linking: static
- go/tags: cmount

[ncw]

This certainly sounds like a bug in something!

Let's concentrate on rclone copy since that has the least extra stuff going on. Can you describe how you are seeing the problem? Ideally provide a little recipe for me to follow to see if I can replicate locally.

With the rclone copy replication, can you see if either or both of these flags help?

  --local-no-preallocate                Disable preallocation of disk space for transferred files
  --local-no-sparse                     Disable sparse files for multi-thread downloads