Ransomware-proof backups (cloud storage without delete permission)

[MaartenBaert]

Automated backups are the best defense against ransomware, however in order to make automated backups possible, the credentials for the cloud storage server need to be stored on the same computer for the backup software to use, which means that ransomware can also use those same credentials to encrypt or delete the entire backup.

Some services (e.g. Backblaze) provide options to generate API keys with limited permissions. For example, with Backblaze I can create an API key that allows file creation, replacement, and 'hide' operations, but blocks delete operations. Since Backblaze keeps old versions of all files, this should make the storage immune to ransomware. There is simply no way to permanently delete the data. Backblaze also has an option to automatically delete hidden files after a fixed amount of time, which should prevent unlimited growth.

Would it be possible to provide an option in Duplicacy to utilize such a storage backend with reduced permissions? I realize that some operations will probably not be possible, but if the important tasks like backup, prune and check are possible with reduced permissions, that should be enough for automation purposes. I don't really need lock-free deduplication, because I'm already using separate buckets with separate credentials and encryption keys for each machine that needs to be backed up anyway, so there will never be two operations on the same bucket at the same time.

[uj]

b2_delete_file_version and b2_hide_file have the same command syntax, so it should just be a matter of replacing "b2_delete_file_version" with "b2_hide_file" in the duplicacy source (or more appropriately, making a command line option for that).

Actually, looking at "duplicacy_b2client.go", HideFile and DeleteFile are already both defined. You probably just need to modify the file "duplicacy_b2storage.go" and change 3 instances of "storage.clients[threadIndex].DeleteFile" to "storage.clients[threadIndex].HideFile" (or more appropriately, making a command line option for that).

Like you said, you can then use specific key permissions in b2, create a key without the "deleteFiles" permission (and other deletion functions). "b2_hide_file" does not require that permission. Then, let b2 take care of deleting hidden files after xx days.

The only thing that I'm not sure about is how you would easily recover those hidden files. I mean, I know how to recover them 1 at a time with the b2 cli, but I've never looked into how to recover thousands of hidden files programatically with b2.

[MaartenBaert]

One problem is that the only way to 'unhide' a file is apparently by deleting the hide marker. Ironically this is not possible without delete permission. So that will require a separate API key.

Another potential problem is that duplicacy uses the delete function also to restore fossils, but I'm not sure under which conditions this happens. If I run prune with -exclusive, is that sufficient to prevent restoring of fossils? What happens with -collect-only? Does it just collect more fossils or does it also restore earlier ones that are still needed?

Suppose that I wrote a tool that restores the entire bucket to an earlier state, by deleting all hide markers and other modifications that have happened after a certain time, would this be a safe way to restore the repository? The downside is that this operation can't be undone ...

Would duplicacy be able to restore a backup while the files are in a hidden state? I assume that this is already supported for data chunks (fossils) but does it also work for the other files (metadata)?

[MaartenBaert]

BackBlaze has announced a new 'copy file' API:

https://www.backblaze.com/b2/docs/b2_copy_file.html

This should make it possible to undelete a file by making a copy of the old version of the file (assuming that B2 deduplicates the data on their end) rather than by deleting the hide marker. It also makes it possible to rename files without downloading and reuploading them, which could be used as an alternative way to mark fossils.

Would that be enough to support the basic operations without requiring delete permissions?

[jderose9]

I would also like to see this in b2. It seems that hiding instead of deleting would provide the most flexibility.