Search for files inside a vault

[tobihagemann]

This is going to be challenging. You can't just search for an encrypted filename in the cloud. Because of our encryption scheme, the filename is dependent on the directory, in which the file is located.

Suggestion Perform a client-side search instead of a server-side one. Create a local cache / search index of cleartext filenames and do a search on that.

Method 1: Create the index, when the user fetches the contents of a directory.

• This obviously doesn't work, if the user has never navigated through the affected folders, or if the file he/she's looking for has been newly created on another device.

Method 2: Background process dedicated to building the index.

• It's a waste of bandwidth. Isn't 100% accurate either, because the user would have to wait for the background process to complete. And we don't even know how often to call this background process.

Can't think of more methods at the moment. I think doing a client-side search with some sort of index is the only way. But it's hard to do without increasing the app's complexity. I'm not amused. :grin:

[monomadic]

I'd actually like for the option to allow for non-obfuscated filenames... I don't really care if people can read a filename, or if a cloud provider knows the filename of my files, especially if it gives me the ability to search.

Another idea though, that you may want to explore, is dumping the directory contents regularly to a text file, then searching the text file. Just open a shell and cd into your unencrypted filesystem (webdav share, this only works on desktop unfortunately). Then:

find . > dir.txt

Then you can either search by opening the text file in a text editor, or you can just:

cat dir.txt | grep <search term>

The value of piping to a text file is you can search the entire structure even when your vaults aren't open, and you can store it unencrypted in your dropbox and then dropbox's search mechanism will search inside the text file for you.

[tobihagemann]

Disabling filename encryption would affect the encryption scheme. We have a feature request and discussion open here: https://github.com/cryptomator/cryptomator/issues/101

Let's say we would keep filename encryption enabled, your other suggestion would purposely leak encrypted information. :smile: That's contradictory to one of the goals of Cryptomator: Encrypt and obfuscate as much (meta)data as possible. That doesn't mean we're not open for changes. E.g., we dropped file size obfuscation some time ago, because the usability benefits overweighed the security concerns.

Reading my original post, I still think that there is no other way around than implementing some client-side solution. Maybe we could make it more effective by implementing it in all our apps (including desktop), so that the search index is saved as an encrypted metadata file. But as soon as you have files that can be concurrently written from different sources, you have to deal with synchronization conflicts.

[monomadic]

I agree, it leaks encrypted information, the filenames - and as I stated those aren't important (to me) if that information is public.. but I understand sometimes that's not the case. You could also store the txt file I mentioned above in your vault, and grep it but that doesn't really solve the problem entirely.

I think perhaps indexing the files might be more trouble than it's worth! especially on platforms where the filesystem is not local (eg. apps like cyberduck and the iOS app).

[wangkesen]

I don't think it's actually that big of deal if you do a good job at organization when you first create the vault got about 700GB oF data on the Webdav now and it isn't an issue. But when I just had one folder and threw all the files in together the vault would take forever to mount as well. Buy definitely better on the new pre release beta.

[ArtificialTruth]

I have been reading the discussions and new feature requests that has sprung out of this issue (https://github.com/cryptomator/cryptomator/issues/336 https://github.com/cryptomator/cryptomator/issues/101 https://github.com/cryptomator/cryptomator/issues/620). It seems first, we need to decide between 2 options:

1. Use the cloud providers search tool (which is usually very good and getting better over time) and sacrifice a bit of privacy for usability by exposing filenames. This also opens up for easy use of the cloud providers versioning tools.
2. Build our own search tool

If we go with 2), one could either: a) Create a cloud encrypted index file (over folders and files) which is updated by all Cryptomator software as often as possible. Handling file/version conflicts should be equally difficult to handle as all the other files synchronized that can be accessed at the same time by Cryptomator software from different platforms, since the index file is only edited by Cryptomator. b) Create a local index, which will be updated every time Cryptomator syncs.

Either way, I think searching is often what prevents people from switching to privacy minded software. For instance I switched to ProtonMail from Gmail, and the search function is missing email content search.

I put my hopes that you talented people will find a solution. Either way, I am happy for Cryptomator :)

[t0ma5]

I think searching is often what prevents people from switching to privacy minded software. For instance I switched to ProtonMail from Gmail, and the search function is missing email content search.

That's exactly why I came to read this issue, I was trying to find how to search for files inside my vault, without it Cryptomator is not an option for me :(

[StanoRiga]

1. Use the cloud providers search tool (which is usually very good and getting better over time) and sacrifice a bit of privacy for usability by exposing filenames. This also opens up for easy use of the cloud providers versioning tools.

For me (and I guess a lot of other users) it is not just a bit of privacy if filenames are exposed. This would mean the provider knows where I am insured, and against which risks (and against which not), where I am going to holidays (and which airline I prefer for example), if i have done my tax declaration (and since when I am doing this), and many many more sensitive information about me. Its not just "a bit" of privacy if one can read my file names.

[patrickwolf]

Yes I totally agree that is's not a bit of privacy I'm using Cryptomator because of the file name encryption (as i have already other means of encrypting file content through my backup program