acd_cli icon indicating copy to clipboard operation
acd_cli copied to clipboard

Published 20 hours ago • verified publisher icon yadayada

Files get truncated after rename via encfs mount (lost data!)

Open Thinkscape opened this issue 8 years ago • 12 comments

Files get truncated when they are renamed on encfs mount on top of acd_cli mount. I've been scratching my head as to why a write is performed at all when renaming.

Use case:

$ acd_cli mount /mnt/acd
$ encfs /mnt/acd/encrypted /mnt/decrypted
$ echo foo > /mnt/decrypted/foo.dat
$ mv /mnt/decrypted/foo.dat /mnt/decrypted/bar.dat
$ stat /mnt/decrypted/bar.dat
 File: '/mnt/decrypted/bar.dat'
 Size: 0

Here's what acd_cli reports at the moment of invoking mv

(ignore changes to xattrs as they are part of #374 )

16-08-23 19:50:41.658 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted (None,)
16-08-23 19:50:41.659 [DEBUG] [acdcli.acd_fuse] - <- getattr {'st_atime': 1471974641.6598833, 'st_ctime': 1470907035.185, 'st_mtime': 1471930081.096, 'st_mode': 16895, 'st_nlink': 1}
16-08-23 19:50:41.660 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.662 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.663 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.665 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.666 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/YoiK4YazOY5xufrAFkPU45OW (None,)
16-08-23 19:50:41.668 [DEBUG] [acdcli.acd_fuse] - <- getattr {'st_atime': 1471974641.6684122, 'st_mtime': 1471967426.746, 'st_mode': 33206, 'st_size': 20, 'st_nlink': 1, 'st_ctime': 1471967425.882}
16-08-23 19:50:41.669 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.671 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.671 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.673 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.674 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.676 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.676 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.678 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.679 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.681 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.682 [DEBUG] [acdcli.acd_fuse] - -> open /encrypted/YoiK4YazOY5xufrAFkPU45OW ('0x8002',)
16-08-23 19:50:41.682 [DEBUG] [acdcli.acd_fuse] - <- open 6
16-08-23 19:50:41.682 [DEBUG] [acdcli.acd_fuse] - -> read /encrypted/YoiK4YazOY5xufrAFkPU45OW (4096, 0, 6)
16-08-23 19:50:41.682 [DEBUG] [acdcli.api.content] - chunk o 0 l 524288000
16-08-23 19:50:41.683 [DEBUG] [acdcli.api.backoff_req] - Retry 0, waiting -14.507516s
16-08-23 19:50:41.683 [INFO] [acdcli.api.backoff_req] - GET "https://content-na.drive.amazonaws.com/cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content"
send: b'GET /cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content HTTP/1.1\r\nHost: content-na.drive.amazonaws.com\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: acdcli.api/0.9.0 python-requests/2.11.0\r\nAccept: */*\r\nConnection: keep-alive\r\nRange: bytes=0-524287999\r\nAuthorization: Bearer SNIP \r\n\r\n'
reply: 'HTTP/1.1 206 Partial Content\r\n'
header: Accept-Ranges header: Cache-Control header: Content-Disposition header: Content-Range header: Content-Type header: Date header: ETag header: Server header: x-amzn-RequestId header: Content-Length header: Connection 16-08-23 19:50:41.803 [DEBUG] [requests.packages.urllib3.connectionpool] - "GET /cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content HTTP/1.1" 206 20
16-08-23 19:50:41.803 [DEBUG] [acdcli.api.backoff_req] - x-amzn-RequestId: 384a23b5-733a-48cb-a8c4-6eb68e825fce
16-08-23 19:50:41.804 [DEBUG] [acdcli.acd_fuse] - <- read 20
16-08-23 19:50:41.804 [DEBUG] [acdcli.acd_fuse] - -> getxattr /encrypted/YoiK4YazOY5xufrAFkPU45OW ('security.capability',)
16-08-23 19:50:41.805 [DEBUG] [acdcli.acd_fuse] - <- getxattr '[Errno 61] No data available'
16-08-23 19:50:41.805 [DEBUG] [acdcli.acd_fuse] - -> write /encrypted/YoiK4YazOY5xufrAFkPU45OW (8, 0, 6)
16-08-23 19:50:41.805 [DEBUG] [acdcli.acd_fuse] - <- write 8
16-08-23 19:50:41.806 [DEBUG] [acdcli.acd_fuse] - -> flush /encrypted/YoiK4YazOY5xufrAFkPU45OW (6,)
16-08-23 19:50:41.806 [DEBUG] [acdcli.acd_fuse] - <- flush None
16-08-23 19:50:41.806 [DEBUG] [acdcli.acd_fuse] - -> release /encrypted/YoiK4YazOY5xufrAFkPU45OW (6,)
16-08-23 19:50:41.806 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:41.807 [DEBUG] [acdcli.api.backoff_req] - Retry 0, waiting 0.157546s
16-08-23 19:50:41.808 [DEBUG] [acdcli.acd_fuse] - <- getattr '[Errno 2] No such file or directory'
16-08-23 19:50:41.965 [INFO] [acdcli.api.backoff_req] - PUT "https://content-na.drive.amazonaws.com/cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content"
16-08-23 19:50:41.966 [DEBUG] [acdcli.api.backoff_req] - <MultipartEncoder: {('content', ('file.bin', <acdcli.api.content._TeeBufferedReader object at 0x7fc59bff4f60>, 'application/octet-stream'))}>
send: b'PUT /cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content?suppress=deduplication HTTP/1.1\r\nHost: content-na.drive.amazonaws.com\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: acdcli.api/0.9.0 python-requests/2.11.0\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Type: multipart/form-data; boundary=540bc03ee9734f4da23c2b5a5470adb9\r\nContent-Length: 195\r\nAuthorization: Bearer SNIP
send: <MultipartEncoder: {('content', ('file.bin', <acdcli.api.content._TeeBufferedReader object at 0x7fc59bff4f60>, 'application/octet-stream'))}>
sendIng a read()able
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type header: Date header: Server header: x-amzn-RequestId header: Content-Length header: Connection 16-08-23 19:50:42.508 [DEBUG] [requests.packages.urllib3.connectionpool] - "PUT /cdproxy/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/content?suppress=deduplication HTTP/1.1" 200 592
16-08-23 19:50:42.509 [DEBUG] [acdcli.api.backoff_req] - x-amzn-RequestId: 830e2523-e6b8-4d89-8c10-1bc13454a339
16-08-23 19:50:42.519 [INFO] [acdcli.cache.sync] - Inserted/updated 1 file(s).
16-08-23 19:50:42.535 [INFO] [acdcli.cache.sync] - Parented 1 node(s).
16-08-23 19:50:42.535 [INFO] [acdcli.cache.sync] - Applied properties to 1 node(s).
16-08-23 19:50:42.536 [DEBUG] [acdcli.acd_fuse] - <- release None
16-08-23 19:50:42.537 [DEBUG] [acdcli.acd_fuse] - -> rename /encrypted/YoiK4YazOY5xufrAFkPU45OW ('/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ',)
16-08-23 19:50:42.538 [DEBUG] [acdcli.api.backoff_req] - Retry 0, waiting 0.776595s
16-08-23 19:50:43.316 [INFO] [acdcli.api.backoff_req] - PATCH "https://cdws.us-east-1.amazonaws.com/drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg"
16-08-23 19:50:43.316 [DEBUG] [acdcli.api.backoff_req] - {"name": "nMs3ay5sD8xHBgnF71zUxKxZ"}
send: b'PATCH /drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg HTTP/1.1\r\nHost: cdws.us-east-1.amazonaws.com\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: acdcli.api/0.9.0 python-requests/2.11.0\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 36\r\nAuthorization: Bearer SNIP 
send: b'{"name": "nMs3ay5sD8xHBgnF71zUxKxZ"}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server header: Date header: Content-Type header: Content-Length header: Connection header: x-amzn-RequestId header: Content-Encoding header: Cache-Control header: ETag header: Pragma header: Vary 16-08-23 19:50:43.547 [DEBUG] [requests.packages.urllib3.connectionpool] - "PATCH /drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg HTTP/1.1" 200 343
16-08-23 19:50:43.548 [DEBUG] [acdcli.api.backoff_req] - x-amzn-RequestId: 1ce4d2a2-695a-11e6-9902-cf327d114bdb
16-08-23 19:50:43.552 [INFO] [acdcli.cache.sync] - Inserted/updated 1 file(s).
16-08-23 19:50:43.568 [INFO] [acdcli.cache.sync] - Parented 1 node(s).
16-08-23 19:50:43.568 [INFO] [acdcli.cache.sync] - Applied properties to 1 node(s).
16-08-23 19:50:43.568 [DEBUG] [acdcli.acd_fuse] - <- rename None
16-08-23 19:50:43.569 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted (None,)
16-08-23 19:50:43.570 [DEBUG] [acdcli.acd_fuse] - <- getattr {'st_atime': 1471974643.5708506, 'st_ctime': 1470907035.185, 'st_mtime': 1471930081.096, 'st_mode': 16895, 'st_nlink': 1}
16-08-23 19:50:43.571 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:43.572 [DEBUG] [acdcli.acd_fuse] - <- getattr {'st_atime': 1471974643.5724294, 'st_mtime': 1471967443.437, 'st_mode': 33206, 'st_size': 8, 'st_nlink': 1, 'st_ctime': 1471967425.882}
16-08-23 19:50:43.572 [DEBUG] [acdcli.acd_fuse] - -> utimens /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ ((1471974641.0, 1471967426.0),)
16-08-23 19:50:43.573 [DEBUG] [acdcli.api.backoff_req] - Retry 0, waiting 0.300848s
16-08-23 19:50:43.874 [INFO] [acdcli.api.backoff_req] - PUT "https://cdws.us-east-1.amazonaws.com/drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/properties/acd_cli_oa-AQ5N1A1KOV6RX/xattrs"
16-08-23 19:50:43.875 [DEBUG] [acdcli.api.backoff_req] - {"value": "{\"fuse.mtime\": 1471967426.0}"}
send: b'PUT /drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/properties/acd_cli_oa-AQ5N1A1KOV6RX/xattrs HTTP/1.1\r\nHost: cdws.us-east-1.amazonaws.com\r\nAccept-Encoding: gzip, deflate\r\nUser-Agent: acdcli.api/0.9.0 python-requests/2.11.0\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 43\r\nAuthorization: Bearer SNIP
send: b'{"value": "{\\"fuse.mtime\\": 1471967426.0}"}'
reply: 'HTTP/1.1 201 Created\r\n'
header: Server header: Date header: Content-Type header: Content-Length header: Connection header: x-amzn-RequestId header: Content-Encoding header: Cache-Control header: Pragma header: Vary 16-08-23 19:50:44.000 [DEBUG] [requests.packages.urllib3.connectionpool] - "PUT /drive/v1/nodes/ZD4JnnleQYm0i8Nm_l6Xcg/properties/acd_cli_oa-AQ5N1A1KOV6RX/xattrs HTTP/1.1" 201 185
16-08-23 19:50:44.001 [DEBUG] [acdcli.api.backoff_req] - x-amzn-RequestId: 1d338f26-695a-11e6-848f-87de8dd43883
16-08-23 19:50:44.005 [DEBUG] [acdcli.acd_fuse] - _xattr_write_and_sync: node: ZD4JnnleQYm0i8Nm_l6Xcg xattrs: {"fuse.mtime": 1471967426.0}:
16-08-23 19:50:44.006 [DEBUG] [acdcli.acd_fuse] - <- utimens 0
16-08-23 19:50:44.006 [DEBUG] [acdcli.acd_fuse] - -> getattr /encrypted/nMs3ay5sD8xHBgnF71zUxKxZ (None,)
16-08-23 19:50:44.007 [DEBUG] [acdcli.acd_fuse] - <- getattr {'st_atime': 1471974644.0069308, 'st_mtime': 1471967426.0, 'st_mode': 33206, 'st_size': 8, 'st_nlink': 1, 'st_ctime': 1471967425.882}

Here's what encfs -d reports at the moment of invoking mv:

unique: 41, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 20518
LOOKUP /bar.dat
getattr /bar.dat
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 2820286894507774090, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (encfs.cpp:133) getattr /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (encfs.cpp:136) getattr error: No such file or directory
   unique: 41, error: -2 (No such file or directory), outsize: 16
unique: 42, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 20518
LOOKUP /foo.dat
getattr /foo.dat
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 4856269329393791124, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/YoiK4YazOY5xufrAFkPU45OW
19:50:41 (encfs.cpp:133) getattr /mnt/acd/encrypted/YoiK4YazOY5xufrAFkPU45OW
   NODEID: 3
   unique: 42, success, outsize: 144
unique: 43, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 20518
LOOKUP /bar.dat
getattr /bar.dat
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 2820286894507774090, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (encfs.cpp:133) getattr /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (encfs.cpp:136) getattr error: No such file or directory
   unique: 43, error: -2 (No such file or directory), outsize: 16
unique: 44, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 20518
LOOKUP /bar.dat
getattr /bar.dat
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 2820286894507774090, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (encfs.cpp:133) getattr /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (encfs.cpp:136) getattr error: No such file or directory
   unique: 44, error: -2 (No such file or directory), outsize: 16
unique: 45, opcode: RENAME (12), nodeid: 1, insize: 72, pid: 20518
rename /foo.dat /bar.dat
19:50:41 (DirNode.cpp:521) rename /mnt/acd/encrypted/YoiK4YazOY5xufrAFkPU45OW -> /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (RawFileIO.cpp:164) getAttr error on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ: No such file or directory
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 2820286894507774090, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (MACFileIO.cpp:71) fs block size = 1024, macBytes = 8, randBytes = 0
19:50:41 (FileNode.cpp:116) calling setIV on (null)
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 0, new IV = 4856269329393791124, fileIV = 0
19:50:41 (DirNode.cpp:641) created FileNode for /mnt/acd/encrypted/YoiK4YazOY5xufrAFkPU45OW
19:50:41 (DirNode.cpp:615) renaming internal node /mnt/acd/encrypted/YoiK4YazOY5xufrAFkPU45OW -> /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (FileNode.cpp:116) calling setIV on /mnt/acd/encrypted/nMs3ay5sD8xHBgnF71zUxKxZ
19:50:41 (CipherFileIO.cpp:94) in setIV, current IV = 4856269329393791124, new IV = 2820286894507774090, fileIV = 0
19:50:41 (RawFileIO.cpp:112) open call for writable file
19:50:41 (RawFileIO.cpp:131) open file with flags 2, result = 4
19:50:41 (CipherFileIO.cpp:192) reading existing header, rawSize = 20
19:50:41 (CipherFileIO.cpp:235) initHeader finished, fileIV = 2459468600337854817
19:50:41 (CipherFileIO.cpp:249) writing fileIV 2459468600337854817
   unique: 45, success, outsize: 16

Thinkscape avatar Aug 23 '16 19:08 Thinkscape

@a5m0 If you've got any additional information on when does this occur for you, feel free to chip in here.

Thinkscape avatar Aug 23 '16 19:08 Thinkscape

I tried doing some bulk renaming of acd_fuse mounted files using FileBot... but all the files got destroyed. This was quite some time ago, 6+ months so older version of acd_cli.

a5m0 avatar Aug 24 '16 02:08 a5m0

I've bulk renamed things a number of times via the rename command successfully: The versions I used were the perl version and the unix commnad version, both in the Ubuntu repos. Perhaps it has to do with the app that did the renaming?

EldonMcGuinness avatar Aug 24 '16 16:08 EldonMcGuinness

My reproduce code uses /bin/mv :\

Thinkscape avatar Aug 24 '16 16:08 Thinkscape

@EldonMcGuinness what's your .encfs config ?

Thinkscape avatar Aug 24 '16 16:08 Thinkscape 

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="10">
<cfg class_id="0" tracking_level="0" version="20">
        <version>20100713</version>
        <creator>EncFS 1.7.4</creator>
        <cipherAlg class_id="1" tracking_level="0" version="0">
                <name>ssl/aes</name>
                <major>3</major>
                <minor>0</minor>
        </cipherAlg>
        <nameAlg>
                <name>nameio/block</name>
                <major>3</major>
                <minor>0</minor>
        </nameAlg>
        <keySize>192</keySize>
        <blockSize>1024</blockSize>
        <uniqueIV>0</uniqueIV>
        <chainedNameIV>0</chainedNameIV>
        <externalIVChaining>0</externalIVChaining>
        <blockMACBytes>0</blockMACBytes>
        <blockMACRandBytes>0</blockMACRandBytes>
        <allowHoles>1</allowHoles>
        <encodedKeySize>44</encodedKeySize>
        <encodedKeyData>
CLIPPED
        </encodedKeyData>
        <saltLen>20</saltLen>
        <saltData>
CLIPPED
        </saltData>
        <kdfIterations>293148</kdfIterations>
        <desiredKDFDuration>500</desiredKDFDuration>
</cfg>
</boost_serialization>

@Thinkscape I just tried mv as well and renamed a file 3 times and no issues with the file.

EldonMcGuinness avatar Aug 24 '16 16:08 EldonMcGuinness

It seems that we differ in the following:

  • I'm on newer EncFS 1.8.1 while you're on 1.7.4
  • I've got newer nameAlg.major of 4, while you have 3
  • I'm using both uniqueIV and chainedNameIV, while you don't.
  • blockMACBytes = 8 while 0 in yours.
  • different key size

Can someone help digest that? Is it possible the newer name algorithm 4 or chaining is causing this?

Thinkscape avatar Aug 24 '16 16:08 Thinkscape

I know I had to use the settings I have to prevent offset errors. Adding uniqueIV and chainedNameIV cause extra bits to be written to the file.

When it comes to the key size, I used 192 due to the fact that 192 is the key size needed for reverse mounting of filesystems with encfs.

I would start by setting the uniqueIV, chainedNameIV and blockMACBytes to match mine. That is what I'm betting is the culprit, but that is just a shot in the dark of course.

EldonMcGuinness avatar Aug 24 '16 16:08 EldonMcGuinness

@EldonMcGuinness i'm testing everything out now. Thanks for tips.

You're on the money with the additional bytes written out of order, which upsets acd_cli mount. Interestingly, with the new encfs settings, I'm able to rsync data into the mount even without PR #374. Renames also work well.

Thinkscape avatar Aug 26 '16 11:08 Thinkscape

Glad I could help! Looks like this can be closed.

EldonMcGuinness avatar Aug 26 '16 16:08 EldonMcGuinness

That ain't fixed by a long shot. This may be one of many different use-cases where specific order of operations causes data loss. In the meantime, we can throw in a few lines into readme. I'll think about it ...

Thinkscape avatar Aug 28 '16 12:08 Thinkscape

I'd see if waiting a few seconds does anything after a file release...

--- a/acdcli/acd_fuse.py
+++ b/acdcli/acd_fuse.py
@@ -334,6 +334,7 @@ class WriteProxy(object):
         if f:
             try:
                 f.close()
+                sleep(3)
             except:
                 raise
             finally:

yadayada avatar Aug 28 '16 15:08 yadayada