acd_cli icon indicating copy to clipboard operation
acd_cli copied to clipboard

acdcli mount "cp" over encfs =25mbit max

Open jxer opened this issue 8 years ago • 16 comments

so whats the deal here, mount works fine, but any type of cp command maxxes out at 25mbit, using rclone i can do over 500mbit up,

i am running encfs on top,

any way to speed cp commands up?

jxer avatar Nov 03 '16 17:11 jxer

@jxer Have you checked whether your CPU may be at its limit? Try doing a raw encryption speedtest on your local hard drive.

BTW, check out gocryptfs. It's pretty much the successor in spirit to EncFS, since the latter is not actively developed anymore.

xelra avatar Nov 04 '16 15:11 xelra

yea its not me and another user see the same thing, rclone can hit close to 600mbit

not to mention the drives are capable of 300MBps

jxer avatar Nov 04 '16 16:11 jxer

can gocrypt read a encfs file? i really don't want to reencrypt everything

jxer avatar Nov 04 '16 16:11 jxer

@jxer No, it's a completely different encryption solution. It just inherits the philosophy of EncFS.

rclone with encryption? You should really check your CPU usage.

xelra avatar Nov 04 '16 16:11 xelra

that sucks, prolly sticking with this for a while then, are your copy commands full speed?

jxer avatar Nov 04 '16 18:11 jxer

@jxer Yes, I'm on a 100 MBit/s symmetrical line and I get 11.8 MiB/s in the test that I just ran with pv -petra file1 > file2, where file1 is on the local filesystem and file2 on the ACD mount with gocryptfs in between.

xelra avatar Nov 04 '16 19:11 xelra

yea its def something with encfs, cp without going through encfs is full speed close to 700Mbit,

i really don't feel like rencrypting argh...

jxer avatar Nov 04 '16 19:11 jxer

@xelra if you're mounting both acd and gocrtpyfs over fuse, what options are you using when you initialized gocryptfs? I ran into data corruption issues before doing that based on how it was caching the results of read calls through acdcli's fuse.

bgemmill avatar Nov 05 '16 00:11 bgemmill

@bgemmill you have a link to what your talking about before i go and spend all day tomorrow trying to set this up

jxer avatar Nov 05 '16 01:11 jxer

@xelra

this is really not working for me, i can't see any update on acd mount after copying over the gocrypt mount without dropping the acd mount and resyncing and remounting, then everything starts showing up correctly

jxer avatar Nov 05 '16 15:11 jxer

I would also update to the newest version of encfs which is 1.9.1. Most distro come with version 1.8.1 which is about 1 year old now. I pulled from the github and compiled it myself. If anyone is interested, I can post the Ubuntu .deb files.

Most modern CPUs come with AESNI which enable hardware acceleration for encryption. There shouldn't be any noticeable performance penalty when using encryption.

natoriousbigg avatar Nov 06 '16 09:11 natoriousbigg

i am running 1.9.1 now myself and with a change in block size on the creation to 4096, i am seeing close to 100mbit on the copy and overall higher performance across the board, it did require to rebuild the entire mount from scratch though

debian was at 1.7.8.

jxer avatar Nov 06 '16 13:11 jxer

@bgemmill I've not run into any issues with data corruption so far. I'm not using any special options. Using gocryptfs has worked out-of-the-box for me.

Maybe I should mention that I additionally use unionfs-fuse. So all my writes first go to the union fs. Only later I will sync the changes from the upper layer of the unionfs to the lower layer (which is ACD). This way I only have sequential writes.

The "syncing" process is then already between encrypted files. There are no more on-the-fly encrypted files written to ACD. I guess that's why I'm not seeing the issue.

xelra avatar Nov 08 '16 21:11 xelra

It looks like the issues I was seeing with gocryptfs were resolved by the time I tried again with 1.1.1.

bgemmill avatar Nov 09 '16 00:11 bgemmill

@jxer If I want to change block size to 4096 I have to re-encrypt everything?

natoriousbigg avatar Nov 09 '16 02:11 natoriousbigg

@ladude626 yep, everything that is encrypted after you make the change will work, but nothing previous will :-(

im on my 4th attempt now at 7tb, its getting tiring

jxer avatar Nov 09 '16 14:11 jxer