gocryptfs
gocryptfs copied to clipboard
rfjakob
I/O errors and freezes when using encrypted directories residing on SMB3 or WebDAV (davfs2) mounts
When using a gocryptfs encrypted directory on a remote drive, mounted using either SMB3 or WebDAV (via davfs2), some operations lead to errors (davfs2) or infinite blocking (SMB3).
To reproduce:
test.py
#!/usr/bin/python
import os
import shutil
base_path = '/remote-plain'
path = os.path.join(base_path, 'rdiff-backup.tmp.0/high_perms_dir')
os.makedirs(path)
shutil.rmtree(path)
shutil.rmtree(base_path)
- Have some remote storage (i can share credentials to the one i'm using)
- Mount it locally using davfs2 or SMB3 (e.g. at
/remote-mount) - Mount the gocryptfs encrypted directory locally (e.g.
gocryptfs /remote-mount /remote-plain) - Execute test.py with according paths, it will yield an I/O error or infinitely hang
Executing the script on /remote-mount instead of /remote-plain works just fine.
More detailed information on when it fails: https://github.com/rdiff-backup/rdiff-backup/issues/641
Mounting using gocryptfs -d -fg -fusedebug ... shows the following output the moment it gives the error:
09:03:03.092534 rx 348: RMDIR n6 ["hl"] 3b
Rmdir: Renaming gocryptfs.diriv to gocryptfs.diriv.rmdir.14960886151134906865
09:03:03.427221 tx 348: 5=input/output error
So apparently it's the rename operation that fails, not an RM. I find it extremely weird, because it works in some cases but doesn't in others and i can't yet pin down what exactly makes it fail, though it is always reproducible when it fails.
Remote storage is a cloud storage product called Hidrive by company 1&1 / Ionos which you can access with different protocols.
I just did a couple more 'tests' using the script: Apparently i can get three types of errors: (1) "No such file or directory": log (2) "Device busy":
OSError: [Errno 16] Device or resource busy: '/backup/hidrive-plain'
(3) "I/O error":
09:03:03.092534 rx 348: RMDIR n6 ["hl"] 3b
Rmdir: Renaming gocryptfs.diriv to gocryptfs.diriv.rmdir.14960886151134906865
09:03:03.427221 tx 348: 5=input/output error
Setting the path (relative to the mountpoint) to a/b/c/d/e/f reliably gives me error (1), a/b/c/d/e/fffffffffffffff (15 times f) gives error (2) and a/b/c/d/e/ffffffffffffffffffff (20 times f) reliably gives error (3).
Paths that either have a short total length or paths that contain few levels (such as a/b/c) don't seem to cause problems.
Another weird thing is that when i remove the directory the script created after i got an error of type (3), e.g. doing $ rm -r /mountpoint/a, i always get an I/O error as well, but it will always work when i just repeat the command.
Looks like gocryptfs does something that either davfs2 and/or the Hidrive server does not like. Yes if you can share credentials, I will test. You can send to [email protected]
Hmm, this is not so good:
09:55:35.622146 rx 3368: MKDIR n382 {0755 (022)} ["d"] 2b WriteDirIV: Openat: no such file or directory
gocryptfs just created a directory, but cannot open it.
Maybe SMB3 is less bad.
Looks like gocryptfs does something that either davfs2 and/or the Hidrive server does not like. Yes if you can share credentials, I will test. You can send to [email protected]
Sent you a mail. Thanks.
So I was unable to mount this (maybe the password has expired), but have you tried setting
debug most
in /etc/davfs2/davfs2.conf ?
Maybe we see what's going wrong here. The davfs2 project seems to be active acc. to https://savannah.nongnu.org/projects/davfs2 , so there's a chance we could get things fixed.
