gilbertchen
[Feature] Bidirectional sync job type
I envision the UX as an additional job type that can be scheduled, with something like an optional checkbox on the Schedule pop-up saying "Sync with storage", and you can then select one of the locations you've created on the Backup page. This would then periodically restore any new or modified files to your local folder, keeping it in sync with the storage location.
This allows creating jobs for file sync and file backup for the same backup set e.g. I can keep an S3 bucket used by multiple devices in sync, as well as have one device back up the dataset to Backblaze for long-term archiving.
I don't quite understand what you're requesting. I have the feeling you want something along the lines of rsync?
Can you please give more detail of your usecase and expectations?
I haven't used rsync but I assume it's like other file sync programs e.g. Google Drive desktop client enables you to keep the files on two or more devices in sync.
I want to be able to make encrypted backups of a local folder via Duplicacy and store it on GDrive, and my other devices to then automatically restore files so that they have the same local files as the first device.
Other solutions I've seen for encrypted file sync means I have to maintain two sets of encrypted folders on cloud storage: one for the backup (Duplicacy) and another for the file sync. It also means the file sync can't take advantage of the versioning that is already being done by the Duplicacy backups. So mainly this feature is requires a scheduled scan of the local dataset, and downloading the differences from the cloud storage in a reversed manner.
this feature is requires a scheduled scan of the local dataset, and downloading the differences from the cloud storage in a reversed manner.
In that case there's no need for a new duplicacy command: this just requires you to create a script to do the "remote" check and run that on a schedule on every PC.
In the end though: i still think rsync is a better usecase for you: you upload-only from PC1 to Gdrive and you download-only from Gdrive to PCx.
I'm not sure how to write a script which would check the remote in that fashion, which is why I'm requesting the feature.
In the end though: i still think rsync is a better usecase for you: you upload-only from PC1 to Gdrive and you download-only from Gdrive to PCx.
Rsync isn't suitable here for several reasons:
- It doesn't work with cloud storage like GDrive
- This feature refers to two-way file sync, so PC1 (or PCx) can both upload and download from GDrive
- It would require two sets of folders at one location again, but this time on local storage: one encrypted version downloaded via Rsync and another unencrypted version which uses some Duplicacy decrypting command. Rather than a Restore-like command which would directly download & decrypt simultaneously
It doesn't work with cloud storage like GDrive
For that you have rclone, a rsync client: https://rclone.org/drive/ (my bad for not using the correct name from the start)
This feature refers to two-way file sync, so PC1 (or PCx) can both upload and download from GDrive
In case of 2 way sync how should duplicacy handle file modification on multiple machines at the same time? What should the "restore command" do?
- blindly delete what you have on PCx and replace with whatever was last backed up (by whichever other PC) -> but what if this existing version is the "good one" compared to that on the storage?
- do not touch any file which exists locally and in the backup at the same time, just add all the new files and folders -> but what if the version on the storage is the "good one" and you actually wanted it synced on PCx?
I hope it is a bit clearer now from these 2 short examples that this feature is extremely complicated and it is my belief that duplicacy, which is a backup client should not have to make such decisions (or have that logic) as this feature is part of a sync program which has multiple knobs and settings for exactly these particular cases. How would you expect things to happen in the script?
Ah I see, that app looks interesting. It doesn't seem to support bidirectional sync though and it would be worse for backups since data deduplication and versioning are limited by not having chunk-based storage or incremental backups/partial transfers.
In case of 2 way sync how should duplicacy handle file modification on multiple machines at the same time? What should the "restore command" do?
I was thinking along the lines of comparing the last modified timestamp of the encrypted files in the cloud backup against the timestamps of their equivalent local files, then overwriting the older files. This is probably a simplified version of how the Dropbox client works also.
But this dropbox-like handling is still not quite safe because in the case where both files were modified, one of the modifications would actually be lost during the sync. :-/
Isn't that the normal behaviour for most file sync tools though anyway? They don't work like git. I'd be fine with that behaviour to overwrite the oldest.
FolderSync on Android offers a few more options for what to do if there are conflicts:
- Overwrite oldest
- Skip file
- Use local file
- Use remote file
- Treat files as identical
Personally I use only and care about the first option but those could be other options available.
I do something like this. On the backup server, I restore the latest revision in a folder using this script (Windows): https://gist.github.com/cristihcd/00c77441c4d31d1833acc82b81b61d02
