android icon indicating copy to clipboard operation
android copied to clipboard

Enhance offline sync job

Open tobiasKaminsky opened this issue 6 years ago • 6 comments

With next version (3.5.0) we will check every 15 min and if on wifi, we update each downloaded file. This is rather easy as we can detect it via changed eTag.

Locally changed files are currently "ignored", as they should either bei directly uploaded/changed in our app or via doc provider (and thus the changes are also instant).

However, we should think how to improve this, so that a downloaded file get updated, if it is changed locally, without notice of our app. This should happen during the same offline sync job.

@nextcloud/desktop how do you detect these changes? Via inotify and handle them directly? For each downloaded file I could compare the server info (which is unchanged as no etag changed) with the current state of the locally saved file:

  • size -> might not change
  • modificationTimestamp -> might have issues with timezones…
  • (hashSum) -> we do not have this right now

Any other ideas?

Maybe modificationTimestamp is enough. If this differs, we try to synchronize the file. Worst case if we had a timezone change is, that all downloaded files tries to sync, but nothing really changes.

tobiasKaminsky avatar Feb 01 '19 08:02 tobiasKaminsky

GitMate.io thinks possibly related issues are https://github.com/nextcloud/android/issues/285 ([TOP] Enhance sync folders), https://github.com/nextcloud/android/issues/668 (Folder sync), https://github.com/nextcloud/android/issues/1898 (syncing problem), https://github.com/nextcloud/android/pull/2886 (Sync downloaded), and https://github.com/nextcloud/android/issues/1008 (Improve offline support).

nextcloud-android-bot avatar Feb 01 '19 08:02 nextcloud-android-bot

Mario pointed out that we already have crc32 as checksum, so we could use this.

tobiasKaminsky avatar Feb 01 '19 08:02 tobiasKaminsky

As discussed on irc: first check for changed mod timestamp, then verifiy via crc.

tobiasKaminsky avatar Feb 01 '19 08:02 tobiasKaminsky

So this is basically what we already do for auto upload - since timestamp changes quite often (well, more often than I'd like), it makes sense to use crc to validate the changes. Collisions are bound to happen, sure, but it's a significantly lighter alternative to other algos and yet good enough.

mario avatar Feb 01 '19 08:02 mario

Timestamp, size, checksum

However I'd go for md5 or sha1 or something instead of crc. On modern hardware this should be trivial to handle if you have to read the whole file anyway.

rullzer avatar Feb 06 '19 18:02 rullzer

Not sure if it's supposed working: in my setup the upload after local changes doesn't even work when online.

I'm using QuickEdit to edit a text file which I marked as "Download". After saving the file and closing the editor, no syncing happens. I always have to tap "Synchronize" on that specific file manually for it to upload.

I've seen various tickets about this but am unsure if online auto-sync when changed is currently supported or not.

PVince81 avatar Feb 17 '21 12:02 PVince81