cppcryptfs
cppcryptfs copied to clipboard
bailey27
File integrity check / diriv & conf Backups
Maybe I should have posted this here since I use the windows version :-)
https://github.com/rfjakob/gocryptfs/issues/270
Almost all sorted out. Apparently I shall make a feature request here for gocryptfs -fsck function to be implemented in the windows version. What do you think?
It is possible to do an MAC validation of a filesystem by creating a new temporary encrypted filesystem and using Windows file explorer to copy the contents of the encrypted filesystem to the new temporary one.
If there are any MAC errors, then file explorer will display an error dialog telling you which file had the problem and ask you if you want to retry copying the file, skip it, or cancel the whole copy operation.
I would also like mention that both cppcryptfs and gocryptfs treat encryption blocks that contain only zeroes (as opposed to encrypted data that decrypts to all zeros) as valid data and return all zeros for read operations on them even though these all zero blocks don't have a valid MAC.
This is an optimization for supporting sparse files (files with holes in them).
So it would be possible for portions of your data to become zeroed out by a malicious person or faulty hardware and the MAC checking wouldn't notice.
Windows has system call that retrieves all the non-hole byte ranges of a file. It appears to be complicated to use and slow, so I don't use it at runtime. But I would like to make the future fsck feature verify that any blocks read as all zeroes from disk are really coming from holes.
That sounds complicated. But right now I’m using Beyond Compare to verify the local files and the remote files have the same size and timestamp. A CRC check after the fact is way too slow and takes too much time. So anything that would verify the integrity of the encrypted files during upload and storage to the remote would be a massive improvement.
also, maybe I should open a new issue for this: I cannot delete folders from within the mounted unencrypted volume. deleting will result in no error, most of the times the files inside the folder get deleted but never the folder. I have to go through the web interface or through the google drive file stream mount in windows file explorer and delete the encrypted folder there and wait for sync to reflect this in the unencrypted mount.
It might have something to do with ownership since I use multiple accounts to upload but it also wont delete a folder created under the same account I’m trying to delete from.
Could you look into this? How can I help?
Is there a way for the unencrypted mount to show either the file/folder owner (drive account owner) or the corresponding encrypted name? It would make finding things so much easier :-)
I don't have Google drive. I will try to get it and test it.
A wild guess is maybe it's getting hung up on the fact that the gocryptfs.diriv files are read-only.
You can see the corresponding file names when the filesystem is mounted by using the --list command line option. For example, if you have a filesystem mounted on K: and you want to see the names of the files in directory "stuff" you can do
cppcryptfs --list=k:\stuff
If you have spaces you and use quotes, e.g.
cppcryptfs --list="k:\my stuff"
cppcryptfs --list=k:\stuff works just fine. however a much more convenient way would be if the corresponding encrypted file name would be shown as a value in windows file explorer (e.g. like bit rate, resolution etc can be shown). especially for large amounts of files and highly nested folder structures that would be great.
the above is nice to have, however, the impossibility to delete folders from within a mounted, decrypted file system in google drive under windows is seriously hindering progress for me. for now I just leave them be and the ones that really annoy me, I rename (that work just fine), access the encrypted files through the google drive web interface, search for the modified file/folder by date and delete the encrypted file. this removes the stubborn folder. can I help with the delete folder issue?
Hi. I installed Google backup and sync and tried it.
I changed my preferences in Google Backup and sync for removing items to "Remove items everywhere", and when I delete a folder in the mounted, unencrypted drive, it gets deleted both locally and in Google Drive.
Do you have the removing items setting like this?
not that I could find but I assume there is none for Drive File Stream.
https://support.google.com/drive/answer/7638428?hl=en
I’m more and more certain it is a cryptfs issue since I can delete folders through the Drive FS web interface but not the mounted unencrypted drive.
check it out, Drive File Stream has a 30 day free eval.
When I try to get Drive File Stream, Google says that you need to have a work or school Google account to use Drive File Stream. I don't have one. I have only a personal account.
I think they key difference is that Drive File Stream provides a virtual drive to Windows, where the backing store is directly in the cloud. Whereas Google Drive works similar to Dropbox.
So if you put your encrypted filesystem on a a Drive File Stream, then the underlying filesystem is not NTFS, Fat32, or exFat, but rather whatever filesystem Google Drive File Stream is implementing.
I guess Google has written an installable filesystem driver for Windows for Drive File Stream, similar to what Dokany has done. So if you use cppcryptfs with it, then you're running an installable filesystem on top of another installable filesystem.
There's no reason why this couldn't be made to work, but I'm afraid there's a good chance that the problem is in Google Drive Stream, in which case I have no control over fixing it.
Are you running cppcryptfs as Administrator or as your regular user?
Could you try running it both ways (as Administrator and not) if you haven't tried that before?
running it admin or non-admin makes no difference as far as the folder delete issue is concerned.
I think you might be right re: file system on top of file system.
nevertheless, I’d like to work with you on a possible solution. managing my 258TiB drive cloud data could become unmanageable otherwise.
google drive for work or g suite business is free to set up an test for the first 30 days. this should give you complete test access. all you need is a domain you can transfer since this only works on owned domains as far as I know.
if you dont have one, I can give you access to some of mine. the free trial is up on those but I dont mind paying for an extension if it helps testing...
also, and I think that might be a Windows limitation, file/path length cannot exceed 240ish characters. though, google drive can go to way above 30000. I assume there is no way around this when going through the explorer interface?
for whatever reason I cannot get the list command to work in PowerShell anymore
PS G:______ENCRYPTION\GoCryptFS> cppcryptfs --list=a:
Program 'cppcryptfs.exe' failed to run: The specified executable is not a valid application for this OS platform.At line:1 char:1
- cppcryptfs --list=a:\
-
At line:1 char:1
- cppcryptfs --list=a:\
-
+ CategoryInfo : ResourceUnavailable: (:) [], ApplicationFailedException + FullyQualifiedErrorId : NativeCommandFailed
A: is the root unencrypted mounted drive....
It's working for me in PowerShell.
I get the same error message you have if I try to run 64-bit cppcryptfs.exe on 32-bit Windows.
Are you sure you're not doing that?
Very sure it’s a 64bit app on a 64bit OS, yes.
Frankly, I’m only trying to get rid of the System Volume Information folder windows created in the encrypted drive for no apparent reason and I cannot delete this POS folder through the mounted drive (Win wont let me no matter what). I figured if I knew the encrypted name I could delete it through the Google Drive web interface :-)
Are you sure the cppcryptfs.exe you're running in the powershell is coming from the same place (path) as the one you've mounted the drive with?
Also, have you tried to run it in a Windows Command Prompt (cmd.exe) window instead of PowerShell?
I would right-click on the icon you launched cppcryptfs with to mount the drive, and select "properties" and then copy and paste the "target" field with the full path to cppcryptfs into a cmd window or powershell window and try that.
The only other thing I can think of is that once I had just installed Windows in a VM and tried to run a program that required the Microsoft Visual C++ Redistributable package, but I hadn't installed the vc redist, and I got an error (in cmd.exe) like the program was for the wrong cpu architecture.
cppcryptfs doesn't require Visual C++ Redistributable, but Dokany does. Did you install Dokany from DokanSetup_redist.exe?
yes, same path, same file. I uninstalled and reinstalled Dokany again, even uninstalled and reinstalled Google Drive File Stream (since it uses dokan drivers as well) but still nothing.
CMD gives me this
C:\WINDOWS\system32>PS C:\WINDOWS\system32>G:______ENCRYPTION\GoCryptFS\cppcryptfs.exe --list=A:
The process cannot access the file because it is being used by another process.
And PS
PS C:\WINDOWS\system32> G:______ENCRYPTION\GoCryptFS\cppcryptfs.exe --list=A:\
just nothing. Runs command goes straight to command prompt. no error or message.
Notice that if you delete the SystemVolumeInformation directory, windows will just recreate it. It is used by System Restore feature, Indexing service, DLT service and Volume Shadow Copy service. As long as any of these services are running Windows will recreate the directory eventually on the drive.
Sooooo. Checking in to see if we can maybe try to solve being unable to delete Folders created on Google Drive File Stream when using ccpcryptfs. As of now I just cut and paste all folders I want to delete into another folder named Trash - which by now is about 300TB large - and solve the issue this way. But that's not really a solution and I much much rather would like to be able to delete Folders through Windows Explorer on the mounted encrypted drive. I know file system on top of file system stuff but who knows, maybe there's a solution? Otherwise if you tell me what exactly the problem is, I can yell at someone I know in Google's Drive File Stream Team....
Could your issue with Google Drive File Stream maybe be related to this other issue I fixed?
I added a setting to automatically delete desktop.ini files created by Google Drive File Stream.
Here is that issue https://github.com/bailey27/cppcryptfs/issues/62
hm. yes and no. I'm confused. I remembered from your post re: issue 62 that one can update the software used (oh my, sorry) and updated cppcryptfs to 1.04.0.27 which triggered the you need dokany 1.3x thing. so I uninstalled dokany 1.2x and installed 1.3x. then started cppcryptfs without changing any of the settings - so they were like they've been before on the old version I used. Particularly, I did NOT tick the "delete ini" box. I was able to delete folders now, for the most part. Something strange happens every now and then though.
Goggle Drive File Stream tries to restore some of the files and folders deleted. I assumed it was a permission thing since we use 5 different accounts to upload and manage content but share it all among us. I tested my theory but it seems untrue. When I used Account A to create a folder and upload content into it and used Account B to delete it, it still worked, no restore attempt by File Stream.
I really dont know what's causing this and stranger even, despite Drive Stream saying it restored the files from the recycle bin to its original location, the files never appear there (even after login in and out again, force synch, changing users....).
Summary, I can delete folders now but I am unsure what this restoration thing does. in my experience with this encryption solution and GDFS, with occurrences like these one ends up with files and folders that are there (you can see them in the encrypted mounted google drive) but they're not "there" in the unencrypted cppcryptfs mounted drive. and I dont like "ghost" things.
Do we wanna find out what this is or shall I go back to my old way of deleting folders and hope some code update somewhere fixes it?