Same-named files mixed during synchronization

[BigMuscle85]

Hello, I have a problem with ownCloud desktop client. I have a directory with our project's source code backup. It contains subdirectories (each contains *.cpp, *.h, ... files) with project's snapshots, something like:

• Project
  • src-2019-07-10
  • src-2019-07-09
  • src-2019-06-27
  • src-2019-05-23 etc.

This directory is synced on ownCloud storage. From time to time, very rarely, the following error is shown for more files by ownCloud client although the directory has not been touched for an half of year (it has not been touched at all since the first synchronization)

07.01.2020 9:31:46,Project/src-2019-05-23/HWManager (conflicted copy 2019-05-24 123257).h, D:\Projects,Conflict: Server version downloaded, local copy renamed and not uploaded.

If I look into the directory, it seems that some files in directory "src-2019-05-24" has been renamed to "... (conflicted copy...)" and replaced with the same-named file from the different directory. Even the copy on the server is wrong.

I checked my old backup and directory had contained correct files before ownCloud was installed, so it was probably mixed somehow during the first sync.

Expected behaviour: ownCloud should not mix same-named files from the different directories.

Windows 10 Pro x64 (1809) Client: ownCloud 2.6.0 (build 12644) Server: ownCloud 10.1.0 (stable)

[michaelstingl]

@TheOneRing Could you have a look in the next sprint?

[HanaGemela]

I'm a bit confused how come the copy on the server is also wrong. So the local conflict file is correct and the newly synced version from the server is wrong because the server is suddenly wrong?

I checked my old backup and directory had contained correct files before ownCloud was installed, so it was probably mixed somehow during the first sync.

Are we here talking about the client or the server? Also the second paragraph says that the error occurred half a year after the initial sync and here it is suggested that the files got mixed up during the initial sync.

The fact that a conflict file was created should mean both versions were changed - the local version and the version on the server. Maybe when the client was offline or when a new account has been added on the client and an already existing folder has been selected etc.

[HanaGemela]

Oh, today my test system is broken in the very similar way.

Steps I did:

1. Yesterday I've created three folders: 01-01-01, 01-01-02, 01-01-03 with the exact same files (same file names). Some files included a note into what folder they belong, some were empty. And everything was perfectly synced.
2. This morning, after two hours of the client just running in the background I got a message "All files were removed on the server, do you want to keep them?"
3. Selected Yes, keep them

Actual result: Conflict files have been created for files with some content. Copies on the server are correct. Conflicted copies are wrong (some empty, some have wrong content), newly synced version from the server are correct. Expected result: No "All files were removed on the server, do you want to keep them?" message

Logs shared Client: 2.6.1 (build 13096) macOS: 10.15.3 Server: 10.3.2 (stable) in docker

[BigMuscle85]

In fact, I don't know that whether the problem is client or server. What I just did:

a) Directory D:\Projects with many project subdirectories. b) One project subdirectory (let's say D:\Projects\Project1) contains several subdirectories with pattern "src-YYYY-MM-DD" c) ownCloud client was installed in July 2019 and directory D:\Projects was set up to sync to server. d) synchronization was successful but I don't know what really was stored to server e) anything in directory D:\Projects\Project1 has not been touched since July 2019 f) some months later, the mentioned error "Conflict: Server version downloaded, local copy renamed and not uploaded" started to appear. g) current state: remote file on the server is wrong, local file is wrong, the correct local file renamed to "... conflicted copy"... the correct file is missing on the server.

But I don't know whether the client uploaded these files to the wrong directory on server, or whether the server put the file in the wrong directory.

Unfotunately, there seems to be no logs. The last entry in .owncloudsync.log.1 is "Syncrun started 2019-10-04T" and nothing older.

[ogoffart]

@HanaGemela

I got a message "All files were removed on the server, do you want to keep them?"

This means that something went wrong on the server, as all the etag were gone. Also possible that the backup recovery from the client kicks in. I'm curious in how you can get that?

This is not the first time We had problem on the server because in PHP, the filename as interpreted as number or something like that. I don't know if that's the problem here.

I don't know how to debug this problem without concreate reproduction steps. But log at the moment this happened would be interresting to look at.

[HanaGemela]

@ogoffart Logs have been shared

[ogoffart]

I looked at the log, and could not find a problem. Everything in the log is consistant. After you clicked on "Keep" in "All files were removed on the server" message, the client gets in a mode where it will not overwrite data and consider any difference between the client and the server as a conflict. And it find files that are different.

Note this part of the log:

02-05 10:25:07:338 [ info sync.discovery ]:	Processing "01-01-01/f2.txt" | valid: true/true/true | mtime: 1580826024/1580826024/1580833981 | size: 0/0/3 | etag: "1c3dd45e33425f21a3e5c509e0b8e2a3"//"8ed4d64eac78dde26b9af1515b6abe8a" | checksum: "SHA1:da39a3ee5e6b4b0d3255bfef95601890afd80709"//"SHA1:3351d58b1d3bf73122c431502fc94a77abba1edc" | perm: WDNVR//WDNVR | fileid: "00000978ocndiu069cga"//"00002591ocndiu069cga" | inode: 20089179/20089179/ | type: 0/0/0
02-05 10:25:07:338 [ info sync.discovery ]:	Discovered "01-01-01/f2.txt" INSTRUCTION_SYNC OCC::SyncFileItem::Down 0

Both the etag and the checksum are different, (the one stored in the local db vs the one advertised on the server.) So this is a genuine difference and the client will download the content. and put the old content of a file in the conflicted copy.

Inhterresting fact is that the size was 0 localy, and that the mtime is also different on the server.

Copies on the server are correct. Conflicted copies are wrong (some empty, some have wrong content), newly synced version from the server are correct.

That's the very strange part. I would have expected the other way around. The file with the "conflicted copy" in the name has the right content, and the server and new files have the wrong content.

Maybe an earlier sync which is not in the log did the corruption. But i have no idea what could cause such a problem.

If you still have the logs of the previous syncs that would be interresting.

[BigMuscle85]

Today, my ownCloud desktop client was updated to 2.7.2 (build 2626) and this happened again. After update, I got tons of the same conflicts for files that have not been touched to 1.5 year and the files were replaced by the same-named files from different folder.

 09.12.2020 8:19:13,RaspberryPi/Tvrdomer/app-2019-07-09/main (conflicted copy 2019-07-04 134729).cpp,                   D:\Projekty,Conflict: Server version downloaded, local copy renamed and not uploaded.
  09.12.2020 8:19:13,RaspberryPi/Tvrdomer/app-2019-07-09/Utils (conflicted copy 2019-06-28 133859).cpp,                   D:\Projekty,Conflict: Server version downloaded, local copy renamed and not uploaded.
  09.12.2020 8:19:13,RaspberryPi/Tvrdomer/app-2019-07-09/Translator (conflicted copy 2019-07-04 134845).h,                   D:\Projekty,Conflict: Server version downloaded, local copy renamed and not uploaded.
  09.12.2020 8:19:13,RaspberryPi/Tvrdomer/app-2019-07-09/MeasurementScreen (conflicted copy 2019-07-09 115333).cpp,                   D:\Projekty,Conflict: Server version downloaded, local copy renamed and not uploaded.
  09.12.2020 8:19:13,RaspberryPi/Tvrdomer/app-2019-07-09/Measurement (conflicted copy 2019-06-27 133430).cpp,                   D:\Projekty,Conflict: Server version downloaded, local copy renamed and not uploaded.

[BigMuscle85]

I got a new computer. I moved all data from old computer to the new computer. I installed ownCloud desktop client 3.2.1.10355 and set the same directory to be synced with server. Many of my local files were scrambled with some old server copy. Local files renamed to "conflicted copy" version and replaced with some very old version.

I'm giving it up. ownCloud is absolutely unreliable software as it can randomly corrupt your files. I don't understand why it cannot at least ask me what to do in case of conflict.

[TheOneRing]

I'd recommend to use git for your usecase

[BigMuscle85]

Git is not correct solution for our usecase. We need to have our local data mirrored to the server with the synchronization in the background. We used OneDrive before and it was working correctly. Then we bought our own disk storage, installed ownCloud and the problems started to appear.

I think that this problem may be related to issue #9591 that I reported some time ago. The performance during the background synchronization is very poor and it seems that it never finishes. When I look at the server data, there are files that are more than month older than the local copy. And I think that the desktop client sometimes get confused and considers the old server copy to be more recent (although the file dates are correct) and replaces the newer local copy.