Repeated conflict report on my todo list file

[PVince81]

Summary: I have a todo list file called "pomorodo.md" and every time (or often) whenever the mobile app syncs, it reports a conflict on said file.

Actual behaviour

Conflict report on "pomorodo.md". When I tell it to keep both versions and diff them, they're both 100% identical.

Expected behaviour

No conflict.

Steps to reproduce

Not reproducible accurately, but here are things I did with the file which might or might not have created this situation:

1. Installed app "DroidVIM" as I wanted a text editor on Android 9 (and the old "vimtouch" is not compatible)
2. Mark "pomodoro.md" file as "available offline"
3. Wait for sync
4. Later on, open "pomorodo.md" with "DroidVIM"
5. DroidVIM is actually broken, it just drops to a shell. Not sure what it did with the app. Likely a bug in that app.
6. Kill DroidVIM and move on
7. Later on, modify "pomodoro.md" on another machine and let it sync with desktop client (my work machine)
8. Unlock mobile phone, it syncs automatically => conflict on this file
9. Resolve conflict and tell it to either keep server version or both, then move on
10. Continue working on file on work machine, let it sync
11. Unlock mobile phone, it syncs automatically => conflict on this file

Other steps to reproduce

1. Add a text file as av.offline.
2. Use "open with" option with QuickEdit to edit the text and then use close + save to upload the file.
3. Repeat the step 2 several times till seeing the conflict dialog.
4. Save both copies in conflict dialog.

Environment data

Android version: 9

Device model: Nokia 6.1 (2008)

Stock or customized system: Stock

ownCloud app version: observed with 2.9.2, still happening with 2.9.3

ownCloud server version: 10.0.10

Logs

Web server error log

• [ ] TODO at the next opportunity

ownCloud log (data/owncloud.log)

• [ ] TODO at the next opportunity

• [ ] TODO: try opening another file with DroidVIM as above and see if it will cause the same behavior

@davigonz @jesmrec How does the mobile app know there is a file that needs to be reuploaded when opened with an editor ? Is there some staging area ? I wonder whether my adventure with DroidVIM caused something to stay in a pending state which would make the OC app think there is something to upload ?

I haven't tried clearing the cache and data of the app. My intuition tells that if I reset the app from scratch the problem will be gone. I'll leave it be for now to give us a chance to debug.

Would be good to know more about technical details and where I can look on the device: how does the app know what to sync, see above question.

[davigonz]

How does the mobile app know there is a file that needs to be reuploaded when opened with an editor?

@PVince81 the job responsible for available offline files synchronization when the ownCloud app is in background is executed every 15 minutes and compares the last modified time of each local file (set as available offline) with the corresponding file in the ownCloud account to detect whether the ownCloud account needs to synchronize the file or not.

Here is where the magic happens.

Maybe checking the last modified time of the files you are modifying with some text editors help us to debug the problem.

[PVince81]

Note that the text editor is now completely closed and I even rebooted the phone. The conflict still happens.

Will check details later including the files' local mtime.

[davigonz]

Will check details later including the files' local mtime.

Ok, thanks for the info

[jesmrec]

not reproducible in an easy way. moving to backlog ftm... with an eye here.

[jesmrec]

Since Android 2.11, logging option is available in Settings view (5 taps on commit hash number to view it). Maybe you can try to reproduce the conflicts issue and send us some logs to take a look @PVince81. It will be appreciated.

[PVince81]

I have the same error now on another file which I edited recently on desktop which is also marked for offline sync on mobile.

During several background syncs, I get a notification about file conflict even though I never edited this file on mobile before, only viewed.

Last part of the mobile sync log looks like this:

2019/08/22 14:57:03 I: AvOfflineSyncJobService : Requested synchronization for file /clientsync/sub1/sub2/thefile.txt in account [email protected]
2019/08/22 14:57:03 D: OwnCloudClient : Executing PropfindMethod in request with id fac9df3c-4ee6-4812-a714-21ff548eac4a
2019/08/22 14:57:03 D: FileDataStorageManager : Number of files updated with CONFLICT: 1
2019/08/22 14:57:03 I: SynchronizeFileOperation : Synchronizing [email protected], file /clientsync/sub1/sub2/thefile.txt: Synchronization conflict

I checked the server access logs and there are no traces of any conflict response, just two propfinds:

x.x.x.x - vincent - [22/Aug/2019:14:57:03 +0200] "PROPFIND /remote.php/dav/files/vincent/clientsync/sub1/sub2/thefile.txt HTTP/1.1" 207 896 "-" "Mozilla/5.0 (Android) ownCloud-android/2.12-beta.1"
x.x.x.x - vincent - [22/Aug/2019:14:57:04 +0200] "PROPFIND /remote.php/dav/files/vincent/clientsync/sub1/sub2/thefile.txt HTTP/1.1" 207 896 "-" "Mozilla/5.0 (Android) ownCloud-android/2.12-beta.1"

I didn't find anything relevant in owncloud.log apart from the usual token messages.

How does the mobile client determine a conflict if not with a 412 Precondition failed or 409 Confict ?

[PVince81]

I didn't have the logs enabled at the time where I set this file for offline sync. I've done so for another text file now and will observe if this happens again.

[davigonz]

It seems that our background job to synchronize available offline files is doing something wrong.

How does the mobile client determine a conflict if not with a 412 Precondition failed or 409 Confict ?

We have extra logic in https://github.com/owncloud/android/blob/master/owncloudApp/src/main/java/com/owncloud/android/operations/SynchronizeFileOperation.java#L188

Thanks a lot for the details @PVince81

[PVince81]

What if there is a slight drift in the times ? Maybe recheck whether mLocalFile.getLastSyncDateForData() is really set at the very last moment ?

I wonder if there is a possibility through delays or race conditions that could cause the local modification date to be higher than the sync date.

Whenever you first download the file after marking for offline sync, I guess its local modification date on disk would match the time where it was written and might not 100% match the sync date ? Or are you doing a "touch" to adjust it accordingly ?

Often times what also happens is that I first view the file by mistake, then close it quickly and then mark it for offline sync. Maybe there's also a race condition hidden there ?

[davigonz]

Whenever you first download the file after marking for offline sync, I guess its local modification date on disk would match the time where it was written and might not 100% match the sync date ? Or are you doing a "touch" to adjust it accordingly?

We also take into account the modification time from Last-Modified header.

The bug likely started to happen when we replaced the old available offline implementation with the jobs one, in which we use AvailableOfflineSyncJobService.

And I still wonder why we decided to put MainApp.Companion.isDeveloper() condition in here because startSyncOperation(.., ..) is always called and I think it shouldn't if localFile.lastModified() <= fileForAccount.first.getLastSyncDateForData(). Need to recheck that.

not reproducible in an easy way. moving to backlog ftm... with an eye here.

@jesmrec some steps to reproduce just below ⬇️

Often times what also happens is that I first view the file by mistake, then close it quickly and then mark it for offline sync. Maybe there's also a race condition hidden there ?

[PVince81]

What is not clear to me is whether the various checks are implying the server's system time and client's system time to be 100% synchronized, in which case small time drifts could exist ? If not then the problem is likely elsewhere. If yes then it might be possible to manually create such time drifts.

[jesmrec]

I gave a new try, by using Android 7 & Android 9

Other steps to reproduce Add a text file as av.offline. Use "open with" option with QuickEdit to edit the text and then use close + save to upload the file. Repeat the step 2 several times till seeing the conflict dialog. Save both copies in conflict dialog.

With these steps i was not able to reproduce either.

I got the incorrect conflict, with steps similar to the original. The point is that these steps do not always work (5 tries, 3 reproduced)

1. Set a txt file as available offline
2. Edit the file with Android and Desktop client. Desktop client will sync at the moment, Android doesn't (i edited the text file in Android, by opening the file with an external editor)
3. Reopen the app and open the txt file. Solve the conflict with "Server"
4. Edit again the file with Desktop client
5. Reopen the file in Android app

Current: Dialog to fix conflict Expected: New version downloaded, since the file is available offline.

Keep researching...

[jesmrec]

This should be fixed with the sync engine refactor that includes new conflict handling. To be released in 3.0.

Closing, feel free to reopen if reproduced again.