MEGAsync icon indicating copy to clipboard operation
MEGAsync copied to clipboard

MegaSync re-uploads files from computer B after they have been deleted/moved/renamed on computer A

Open RiseT opened this issue 8 years ago • 54 comments

OS: Windows 7, Windows 10, 64 bit MegaSync version: so far all (currently 2.9.6)

In the past, I've experienced this "vicious circle" numerous times in different situations.

  1. Computer A and computer B are fully synched
  2. I delete files or folders on computer A (issue mostly happens if this is a large amount of files/folders)
  3. Sync client of computer A updates the cloud storage
  4. Computer B notices that the files/folders exists locally but not in the cloud
  5. Computer B uploads the "missing" files/folders to the cloud
  6. Computer A notices that there are new files/folders in the cloud and downloads them
  7. I'm where I've started out again.

For one practical example, see https://github.com/cryptomator/cryptomator/issues/304#issuecomment-231455692

RiseT avatar Jul 08 '16 20:07 RiseT

Hello, we have been trying to reproduce the problem but we have been unable. For us Computer B always removes the local files after the step 4.

Are you using the same account in both computers? Or are you syncing a shared folder in one of them?

The usage of Cryptomator could be changing the scenario a bit. If those deletions are done in the virtual drive, the real files created by cryptomator could be doing something different and maybe that confuses MEGAsync. Or maybe cryptomator on Computer B doesn't like that MEGAsync deleted his files at runtime. Anyway, I'm just guessing.

MEGAsync should recognize the external deletion (or rename) and apply it in the local filesystem, except when the synchronization is added to MEGAsync. During the first scan, if MEGAsync detects missing files, it follows the most conservative approach and creates the missing files on the other side instead of deleting them. But once a synchronization is already added (it doesn't matter if MEGAsync is not running or is restarted) MEGAsync should correctly propagate the changes.

If you are able to reproduce the problem. Please create a debug log with Computer B while you are reproducing it. That way, we could see how the external notification is received, how it is managed by MEGAsync and why it doesn't apply it correctly.

To enable de debug mode, start MEGAsync with Ctrl+Shift pressed (Cmd+Shift on OS X), or open the settings dialog and click five times in the shield next to your email address. When the debug mode is correctly started, MEGAsync shows a notification that says that it is creating a log ("MEGAsync.log") in your desktop.

Once you have a log showing the problem, please send it to us to analyze it and fix the problem. If you don't want to share it here (it could contain sensitive data like filenames, etc.), please send it to [email protected] and post here the ticked ID so we can contact our support team to get your log. If you can provide additional information (like the name of the file that was incorrectly managed) that would speed up our investigation. Anyway, once you send your report to support, we could contact you directly by email to keep you updated and request additional information if needed.

Thank you very much!

javiserrano avatar Jul 13 '16 15:07 javiserrano

Hello, I'm facing a similar issue for a quite long time, not just in the last versions. I'm using SyncToy to make a copy of a Thurderbird profile to a folder C:\Email backup, which is synced by Megasync. There was a folder with about 2000 files in the folder extensions in the TB profile. I don't know the reason, but the folder is not there anymore (maybe after TB update). Now everytime I try to upload a new copy of the TB profile backup, Megasync downloads the 2000 files again. This happends on more than one computer. Everytime it's just one computer connected to each account. I have launched Megasync in a debug mode and deleted the folder. After a while, Megasync downloaded all the files again. I'm sending you the log and also the folder, so you can try to reproduce the bug. Just put the folder inside some folders you're syncing and then delete it after the sync. Hope this helps.

extensions.zip

hight0wer avatar Jul 18 '16 00:07 hight0wer

I'm still unable to produce this on purpose, but similiar to @hight0wer , I've been experiencing this once in a while since I've started using Mega 1 or 2 years ago.

I've used to think this only happens when I delete/rename a huge number of files, but yesterday, it happened when just moving a folder with a couple of files slightly deeper down the (sub-)folder structure (unfortunately, I hadn't activated logging at that moment). The result has been a duplicate of that folder. One at its original location (got re-downloaded at the moment MegaSync got started on a second computer) and one at the location I've moved the folder to.

I'll try to activate logging daily now and wait until the next occurence.

PS: Is there a way to activate logging permanently, i. e. logging still activated after a restart of MegaSync/reboot of OS? Otherwise it will be hard to log this since MegaSync on the second computer, which is incorrectly re-uploading the files deleted on the first computer, is autostarted with the OS and will start uploading instantly after scanning. So there's no time to activate logging. I suggest logging of a trailing time window of 24 hours (so anything older gets deleted).

RiseT avatar Jul 18 '16 08:07 RiseT

Hi,

I'm having the same issue, in the opposite direction: sometimes I delete a local file and the MegaSync client downloads it back from the cloud. This happens quite frequently, sometimes more than once. This also happens when I move files or directories in the local machine, which results in having duplicated files everywhere.

My system is Ubuntu 16.04 - 64 bit, using Megasync client version 3.1.4 (0065d1).

In the logfile attached, the culprit files are memoria.txt and lcrandom.h.

MEGAsync.log

cmaziero avatar Nov 04 '17 13:11 cmaziero

Just experienced this again recently with two Windows 10 machines (MegaSync v3.5). As this doesn't happen very often (but when it does, it can get ugly), it's pretty hard to enable logging at the right moment.

I've most often had this when one of the computers hadn't been online & thus synched for a couple of months. If I had deleted (or moved or renamed) some files or folders on the "more up-to-date" machine, bringing online & synching the "outdated" machine happily recreates the deleted files/folders in the cloud (and creates duplicates if the files/folders had been moved/renamed rather than deleted) by uploading them from the outdated machine.

RiseT avatar Nov 07 '17 10:11 RiseT

This problem exists for me too. I'll try to describe it, since I have no option to log it at the moment. This has occured maaaaany times and it's starting to be annoying because we have more users to deal with at the moment. I can't keep up with keeping the drive organised this way.

  • Computer A is Windows 8.1 x64
  • Computer X is always a OS X variant
  • Share a folder with another contact (not the same account) (= computer X)
  • Both computers have MegaSync installed.

I can reproduce it very easily:

  1. Take any number of files, sync/upload them onto your cloud drive
  2. Make the share with the other contact, set it to read-only (he downloads the files)
  3. Change the filename and location of a number of files (in windows explorer)
  4. Set the share to 'full access' with the other contact (computer X)
  5. Let the other contact set up the (shared) folder sync in the settings/SYNCS tab on his computer.
  6. Witness files being downloaded that you renamed or moved earlier.

Note that having the share set to read-only, also means that it could very well be that Computer X is offline. What happens for me very often is that when other Computers with the sync setup have been offline for a day or two, I changed file location, names or deleted some, and they don't receive the new state of these files.

My logical instinct says that there is something wrong with the way MegaSync handles filechanges. It scans for changes first, which happens before it checks if files have been renamed, moved or deleted by another user. Somewhere along the line this data of 'renaming/moving/deleting' gets stuck in the pipe and doesn't get sent through.

Clearing cache / emptying rubbish bin does not help. I tried on both occasions (with or without clearing it).

Good luck with reproducing this. I'll keep watch to help out.

ghost avatar Nov 12 '17 19:11 ghost

I've uploaded the log and files, which can be used for reproducing the bug, more than a year ago. I think it should help to find the problem, so you all don't need to bother with making another logs unless noone uses the informations we've provided..

hight0wer avatar Nov 12 '17 21:11 hight0wer

New in this version:

  • New UI style
  • Support to generate file versions in MEGA
  • Integration with Finder (macOS 10.10+)
  • Allow to exclude specific files/folders
  • Bug fixes and other minor improvements

srsly.

ghost avatar Nov 17 '17 07:11 ghost

Can we put in a joint effort to try and reproduce this bug, and see if it still exists in the new version with the new 'file versions' addition?

ghost avatar Nov 19 '17 22:11 ghost

When I've downloaded the MegaSync client a couple of weeks ago, I got version 3.5 (with which I still experienced this issue). A couple of days ago, though, the MegaSync client updated itself. But the version number is still 3.5. I'm slightly confused about the difference of those two versions.

RiseT avatar Nov 20 '17 10:11 RiseT

Things are worse here: after the last update (3.5, e16ed2), the client stays in a "scanning..." state forever, and syncs are not being done at all. I opened a ticket with Mega suport, let's wait.

cmaziero avatar Nov 20 '17 11:11 cmaziero

That's interesting. I have v3.5, f950d4, and the client says there's no additional update available. I wonder how many builds of v3.5 are floating around...

RiseT avatar Nov 20 '17 11:11 RiseT

I believe the build number depends on the OS. Mine's is Linux Ubuntu 16.04 64bit.

cmaziero avatar Nov 20 '17 11:11 cmaziero

Ah, ok, makes sense. I'm speaking of the Windows client.

RiseT avatar Nov 20 '17 11:11 RiseT

I am using the sync app for Debian 9.0. I have just experienced the same issue, after 10 hours of making changes to one computer, the changes were reversed. Some logging in and out might have been involved ruing the changes were made, also, I had different versions of the sync client on both computers (one was 3.5, the other someting like 3.1.4 I guess.). Btw., is there any log of changes on the website, so that I can revert the automatic changes more easily?

aliulo avatar Nov 29 '17 10:11 aliulo

This is a nasty bug if it hits you, creating many hours of unnecessary work...

RiseT avatar Nov 29 '17 11:11 RiseT

v3,5.3 says "Improved the management of deleted files". Don't know if this is related to this issue as that version isn't on GitHub yet.

RiseT avatar Dec 02 '17 10:12 RiseT

Is there anyway to force megasync to use the local folder as reference. I am having the same issue as @cmaziero

noxecane avatar May 24 '18 19:05 noxecane

Has anyone tried this lately, i. e. has this been fixed? I'm hesitant to sync a second computer with my Mega cloud since it is so much work to clean up the mess this bug is creating when it hits.

RiseT avatar Jul 28 '18 11:07 RiseT

Hi RiseT, I'm using it to sync 130 GB of files in 3 computers (all Ubuntu 16.04) these last 3 months and everything seems fine to now (crossing my fingers here...).

cmaziero avatar Aug 06 '18 17:08 cmaziero

I have a similar problem, quite annoying.

  • I upload the photos automatically on my phone to Camera Uploads on Mega.
  • I installed ubuntu on a computer.
  • I copied all the files on a Mega folder from a backup in order to avoid downloading them again.
  • All the files in the "Camera Uploads" folder on the computer are being re-uploaded and then they appear in Rubbish bin on Mega. All names of the folders match

zukunfter avatar Aug 29 '18 22:08 zukunfter

zukunfter, this arrives frequently, because Megasync uses only times/dates and sizes to compare files: if two files are equal but have different dates, they will be synced. Synchronizations would be much more efficient if MS used hashes to compare files, to avoid re-syncing identical files. There is an open issue on this problem, with no answer yet: https://github.com/meganz/MEGAsync/issues/181.

cmaziero avatar Aug 30 '18 11:08 cmaziero

This nasty bug was not yet solved...

Yesterday I deleted/zipped hundreds of old files and directories in my home PC. Everything seemed ok. However, this morning the MegaSync client started and, after a quite long "analyzing files" pause, it re-downloaded most of the files I deleted yesterday, including entire directories that I zipped to take less place. :-(

I'm running MegaSync client version 3.6.7 (99a46c) on Linux Ubuntu 16.04.5 in two machines (at work & at home).

cmaziero avatar Sep 10 '18 08:09 cmaziero

@cmaziero Many thanks for this info. Will keep myself from synching another computer and mess up my data. I've done that too often now.

RiseT avatar Sep 10 '18 11:09 RiseT

I have the exact same problem, especially as described in OPs second comment in this thread.

It also happens to me, especially when I haven't used the second computer in a while. It's really annoying because it means that you can't rely on the changes you've made. I will do a manual one-way sync on PCs I use infrequently from now on, it saves a lot of time compared to finding duplicates and files that should not be there.

I think it might have something to do with Mega discarding the database/file changes protocol, so after some time, it just treats the second PC as a sync that has been set up for the very first time.

thompson004 avatar Dec 05 '18 16:12 thompson004

I think it might have something to do with Mega discarding the database/file changes protocol, so after some time, it just treats the second PC as a sync that has been set up for the very first time.

That's an interesting theory that is in line with my experience with MegaSync (i. e. it happens if I haven't used the second device for a while).

RiseT avatar Dec 14 '18 10:12 RiseT

The issue occured on my end as well. It seems that if kit 1 removes the file while kit 2 is offline/has app turned off, then when kit 2 opens the app again, the removed files are uploaded again instead of being removed.

Repro steps:

  1. Launch MEGASync on kit 1 and kit 2.
  2. On kit 1 share a folder with files with kit 2 and let both kits sync.
  3. Once done close MEGASync app on kit 1.
  4. On kit 2 remove a file from the shared folder and ensure it is removed on the site as well.
  5. Reopen MegaSync on kit 1. Observe how removed file is uploaded to the shared folder again.
  6. On kit 2 observe how the removed file is downloaded again.

This makes it impossible to use MEGA for group projects, like modding communities etc which need a synchronised folder to upload their work. The issue above makes it impossible to remove not needed files, because it will always be uploaded again by another user that wasn't online at the moment of file deletion.

Castigavi avatar Apr 15 '19 19:04 Castigavi

Same issue.

  1. Downloaded file to d disk.
  2. Extracted archive to same disk. Result: Capture

dubluxas avatar May 09 '19 10:05 dubluxas

Same problem between MacOS and Windows clients. For me, the Windows client re uploaded files that I previously deleted in the cloud and in the Mac folder. Files re appeared and lost all the clean up and organization work I did!

JaEdmuva avatar Jul 02 '19 16:07 JaEdmuva

Quite frankly, I don't understand why 3 years after opening this issue, this serious problem still isn't fixed, while at the same time the client is getting other and less critical commits constantly.

RiseT avatar Jul 03 '19 09:07 RiseT