Possible Bug: 100% disk activity on large files?

[heikoh81]

Hi, I noticed a possible bug that I cannot locate. It might come from Dokany or cppcryptfs. It happens if I copy large files (~6-8 GB each) to an encrypted USB Seagate HDD. The first few files write ok (but very slow at 16 MB/s). The more I copy, the slower write speeds get. Disk activity goes up to 100%, Speed drops to 2 MB/s. After several minutes writing is nearly impossible.

This happens with different external drives and on different PCs (1 with Win10 LTSB 1604, 1 with Win 8.1). Dokany is 1.2.0.1000, cppcryptfs is 1.4.0.19. It does not happen with VeryCrypt on the same PCs and the same USB Disks, and speed is 50 MB/s with VeryCrypt. Unencrypted write speed to these external drives is ~ 100-120 MB/s. The USB drives are very new, and have no S.M.A.R.T warnings.

The problem only occurs while writing, never while reading. Reading also performs at 50 MB/s. At the moment I have to go back to VeraCrypt for my large file use scenarios, while I can use gocryptfs with 1000s of small files without performance issues. I'd prefer gocryptfs for the future because it is file based and not container based!

Best regards, Heiko

[bailey27]

I'm sorry but I wasn't able to reproduce this problem.

cppcryptfs should perform the same as VeraCrypt on large files. But VeraCrypt should perform better than cppcryptfs when dealing with lots of small files. You seem to be observing the exact opposite.

I tried using up to ten 8gb files of random data. I tried my C: ssd, my internal mechanical D: drive, an ssd in a usb 3.0 enclosure and a mechanical drive in a usb 3.0 enclosure.

In every case, the performance of cppcryptfs was almost the same as VeraCrypt.

Could you try to reproduce the problem using dokany mirror?

It's a command line program in C:\Program Files\Dokan\Dokan Library-1.2.0\sample\mirror

e.g.

mirror /l x: /r d:\tmp

This would make drive letter X: mirror the contents of directory d:\tmp

It asks to be run as administrator but works anyway without administrator privileges.

[heikoh81]

Thanks for your tests. This is strange, indeed. I even disable Avira FreeAV Realtime Monitoring during copy, to exclude this as a possible cause.

I'd try the mirror command. I have these questions: Does d:\tmp have to exist? Does drive x have to exist? I use the letter X for gocryptfs and VeraCrypt mounts - what if a drive x is already mounted, will data be overwritten? I don't want to lose data...

Best regards, Heiko

[bailey27]

The folder must exist. I gave d:\tmp for an example, but you can use any folder that already exists.

The driver letter should not already be used.

[heikoh81]

I did some further testing.

With Veracrypt, I get Windwos Error 665 if the Container-File (dynamic growth) gets to big (> 1,3 TB). Windows disk analyses shows 99% fragmentation. Although I could not Google any evidence with VeraCrypt, this seems to be a problem on my side. https://blogs.msdn.microsoft.com/psssql/2015/06/10/operating-system-error-665-file-system-limitation-not-just-for-dbcc-anymore/

So my only option is to go with gocryptfs. I reformatted my 6 TB / 8 TB USB-HDDs with exFAT (would prefer ReFS, but not available on Win 10 1607 LTSB). Now all my problems are gone, disk activity maxes at 95%, write speed is up to 120 MB/s with cppcryptfs (average speed is 85 MB/s, though, but still very very good).

So the problem seems to be NTFS, although I always though this is the file system to go on Windows systems.

I'll continue with exFAT on external drives from now on. Another advantage: I can mount those disks on any Linux and use gocryptfs there!

[bailey27]

That's great. exFAT is a good filesystem. You can have paths over 32,000 characters long just like NTFS.

I've updated the Readme.

[ccchan234]

I may got the same problem as OP. But slightly different as I dont have large files (I keep files < 4GB for easy transfer but I got tons of small files)

I still use 1.4.0.15 and likely dokany 1.1x.x (the dokany installed along encfs4win at the moment)

I am copying 4TB of files (<4GB/file) from a truecrypt partition (NTFS) to a cppcryptfs container, on NTFS.

The copy is made using beyond compare 4, and it at some late stage will get halt-ed, at least three times for me, and I have to manually disconnect the target drive from the system.

After reading this post I am now trying to use exfat on the target.

Will report success or fail or if i tried with most updated version of cppcryptfs/dokany.

Thanks.

ps: Dokany v1.1.0.2000 (modern)

updates: 20181225-121133 Using exFAT on the target USB HDD makes the copy overnite possible (which if using NTFS, will halt, I remember it later will show need 5xxxx minutes). And I just found that I indeed got some HUGE files like 100GB which are ghosting drive images. I will try finish this copy using exFAT, and if have time and extra HDD, i will try the latest cppcryptfs and donkay with NTFS. At the moment I will use exFAT as NTFS is not feasible.

Oops.. merry X'mas

20181225-162613 so far, the copying of 4TB to cppcryptfs on exFAT is working well. for cppcryptfs on NTFS, it's not working for 4TB. yet I am currently also copying another 2TB data to another 2TB cppcryptfs on NTFS, it works well however.

20181227-130537 finally thought until have free time to track the problem: As said above, my target is simply to copy files (small and >4GB, could up to 100GB) inside truecrypt partition (as NTFS, is internal SATA 4TB) using beyond compare 4 to a cppcryptfs on a USB HDD. The cppcryptfs if rest on NTFS, display need 5xxxx mins, and halt several times for me. Making the task impossible to finish. Currently it likely to be OK (files are copying) while I modified these:

1. as OP say before, use exFAT on the target drive
2. I updated to 2nd? newest cppcryptfs (1.4.0.21) /w the associated new dokany
3. beyond compare 3/4 is special that it can handle path names > 300, and this time I try to copy the 4TB in several chunks so it may make cpu/memory requirement less.

ps: however, the "impossible config... the NTFS config" do work well for me when it is 2TB and the target is a SATA 2TB.

Not sure will have time to play around the files later, but if other user got the same problem, could try the suggestion and give feedback here, or report the same problem, so this community could have more cases and analysis.

Thank you, and thanks the author for cppcryptfs

[ccchan234]

major update: When I want to copy 4TB files from NTFS on truecrypt partition (on USB raid 1 box, SATA HDD) to cppcfs on exFAT on 4TB USB drive. while I only copied about 1.2TB (by beyond compare 4), it say that the target disk drive is only have 109MB left.

So even I tried this copy movement since 20181225, I still dont have the work done. May be I will try back using NTFS... not sure what's the problem. (I know the encrypted files will be larger than the original files but 1.2TB cant occupy 4TB drive i think)

thanks

target drive: when browsing on the plain exFAT, outside cppcfs: size: 756 GB (812,112,757,755 bytes) size on disk: 909 GB (976,314,433,536 bytes)

drive property: used space: 3.63 TB free space: 109 MB

update: 20181229-085246 used back cppcfs on NTFS, now copying using free file sync v9.6, could copy overnite. (NTFS using "beyond compare", will halt if copy overnite) Let's hope it's beyond compare's problem (but the 1TB file occupied 3.6TB shd not due to beyond compare... Let's trouble shoot later)

[bailey27]

I have no idea why you see a different disk space usage outside of cppcryptfs vs. doing drive property on the mounted cppcryptfs drive letter.

All I do when Dokany asks for the free space amount is to call the Windows API function GetDiskFreeSpaceExW() on the actual drive letter where the encrypted filesystem is. I don't see how the sizes could differ.

I think you should maybe try Robocopy. It is from Microsoft and comes with Windows. It handles very long paths.

It is command line only, though.

It comes with ordinary Windows, not just Windows server.

https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy

https://en.wikipedia.org/wiki/Robocopy

[ccchan234]

thanks for reply.

1. using NTFS + freefilesync seems working overnite so I stopped playing with exFAT. (also stopped using beyond compare to copy)

2. I am still in progress of copying 4TB files into cppcfs on NTFS, will report if finally success (hope so) or not.

Happy new year.

progress: 20190104-004215 everything seems fine. using FFS copy /w verify 4TB files from (NTFS on truecrypt) to (cppcryptfs on NTFS). 2TB have been copied and the space used appear to be normal. (onging) All parameters are same except "beyond compare 4" vs "free file sync" as the program. Same HDD, same PC (although I have updated cppcrypfs from 1.4.0.15 to 1.4.0.21)

[ccchan234]

hi~ it appears the problem could be due to beyond compare 4. And trying to update to 1.4.0.21 may be a solution if anyone still encounters. Mean while thanks author 's concern and think this issue could be closed, thanks. (ps my files got files up to 100GB so the OP's concern is over. there is no problem using cppcryptfs with files up to 1xxGB for me)

[heikoh81]

I still don't think that this problem is Software-related. I had 100% disk activity on copying with Windows Explorer, SpeedCommander, Goodsync and FreeFileSync.

I think that heavy fragmentation takes place on NTFS-drives. To me it seems that this has started with HDDs > 5 TB. And on those drives I normally store large files 4 GB+ and I do linear writing - so fragmentation should be no problem at all. I'm no expert on file systems, just a heavy user. All I have read is that large files or large drives should be no problem for NTFS, but in my experience there were problems on various PCs with various Win versions.

See my link postet above: https://blogs.msdn.microsoft.com/psssql/2015/06/10/operating-system-error-665-file-system-limitation-not-just-for-dbcc-anymore/

I have no problems any more using exFAT. I have not tried ext4, as I don't use my large HDDs with Linux.

[ccchan234]

I still don't think that this problem is Software-related. I had 100% disk activity on copying with Windows Explorer, SpeedCommander, Goodsync and FreeFileSync.

I think that heavy fragmentation takes place on NTFS-drives. To me it seems that this has started with HDDs > 5 TB. And on those drives I normally store large files 4 GB+ and I do linear writing - so fragmentation should be no problem at all. I'm no expert on file systems, just a heavy user. All I have read is that large files or large drives should be no problem for NTFS, but in my experience there were problems on various PCs with various Win versions.

See my link postet above: https://blogs.msdn.microsoft.com/psssql/2015/06/10/operating-system-error-665-file-system-limitation-not-just-for-dbcc-anymore/

I have no problems any more using exFAT. I have not tried ext4, as I don't use my large HDDs with Linux.

Hi I dont know if OP and I got the same problem. Yet just share above that experience while I copy 4TB from truecrypt to cppcryptfs. I cant do that with beyond compare, and it's significantly slower than not using cppcryptfs(due to dokany?), and it can be done with files > 100GB, while I updated to 1.4.0.21 and use free file sync. finally I can stick to NTFS without using exFAT. thanks.