Multiple Folders Duplicated

[balthanon]

Which version of floccus are you using?

5.4.5

How many bookmarks do you have, roughly?

2700

Are you using other means to sync bookmarks in parallel to floccus?

No for bookmarks; I do sync other things like browsing history via Mozilla/Microsoft/Chrome

Sync method

Google Drive

Which browser are you using? In case you are using the phone App, specify the Android or iOS version and device please.

Firefox 136.0.4 (64-bit); Waterfox 6.5.6 (64-bit) branched from Firefox ESR 128; Chrome Version 134.0.6998.178; Vivaldi 7.3.3635.4 (Stable channel) (32-bit) which is branched from Chrome 134

Which version of Nextcloud Bookmarks are you using? (if relevant)

N/A

Which version of Nextcloud? (if relevant)

N/A

What kind of WebDAV server are you using? (if relevant)

N/A

Describe the Bug

A month or two ago, I discovered that my bookmarks had more than doubled-- several folders had been duplicated and added to the bottom of my bookmarks list; including the entire Mozilla Firefox Bookmarks Menu (my entire bookmark list) plus several of the subfolders. I posted a question about it and how I might track down how it had happened; at the time that happened I caught it several weeks after it occurred so I couldn't really narrow down where the problem might have started, but I was eventually able to restore my bookmarks from backup.

The same issue just happened again today though, so it doesn't seem like it was a one off issue and I believe it happened within the past hour or so, while I wasn't even at the computer. I have verified that I have no alternative syncing that I can find active and unless it happened prior to when I think it did, I had only one computer on and 4 browsers open (Firefox, Waterfox, Chrome, and Vivaldi). I do have multiple computers and 6-10 browsers installed on each of them though.

I pulled the full logs, shortly after I found the issue and Waterfox had one line that was different-- I'm not sure if that indicates it is the source of the problem or not:

2025-04-03T23:53:09.965Z Move untouched children back into place { untouchedChildren: [ [ 11, '1mnSbCAh8Qty' ] ] } 2025-04-03T23:53:09.969Z

Another potentially corroborating fact is that I then pulled redacted logs maybe 10 minutes later and the waterfox log is significantly longer than any of the other logs. 61000 lines vs 387 for Firefox. I am having trouble attaching the logs, so I'll try to do that after submitting again.

Expected Behavior

Syncing should not duplicate bookmarks or folders.

I do have a couple of feature suggestions I'm going to submit alongside this bug as well that I think would help in situations like this:

1. Have another failsafe option that prevents the browser from adding more than 50% of the current number of bookmarks in one sync as well.
2. Have a sync change tagged with the browser and computer that it originated from somehow; whether that is pulled from the browser stats or is just a tag that a user can add to a given installation of Floccus. This would make tracking down where an issue originated significantly easier.

To Reproduce

I can't really provide reliable steps to reproduce, as outside of the issue occurring twice this year, I haven't been able to identify why it is occurring or what triggered it. The best I can say is to leave the 4 browsers open while the syncing is occurring-- it does look like several of the browsers were in the process of updating though; Firefox and Chrome are both waiting on restarts for updates.

If there is anything I can provide that would help to track down the issue, I would definitely be happy to provide it though.

Debug log provided

• [x] I have provided a debug log file

[github-actions[bot]]

Hello :wave:

Thank you for taking the time to open this issue with floccus. I know it's frustrating when software causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at and if possible solved. Let me give you a short introduction on what to expect from this issue tracker to avoid misunderstandings. I'm Marcel. I created floccus a few years ago, and have been maintaining it since. I currently work for Nextcloud which leaves me with less time for side projects like this one than I used to have. I still try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it. Until then, please be patient. It helps when you stick around to answer follow up questions I may have, as very few bugs can be fixed directly from the first bug report, without any interaction. If information is missing in your bug report and the issue cannot be solved without it, I will have to close the issue after a while. Note also that GitHub in general is a place where people meet to make software better together. Nobody here is under any obligation to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can collaborate to make this software better. For everyone. Thus, if you can, you could also have a look at other issues to see whether you can help other people with your knowledge and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and try to fix the odd bug yourself. Everyone will be thankful for extra helping hands! If you cannot lend a helping hand, to continue the development and maintenance of this project in a sustainable way, I ask that you donate to the project when opening an issue (or at least once your issue is solved), if you're not a donor already. You can find donation options at https://floccus.org/donate/. Thank you!

One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the Nextcloud forum, to twitter or somewhere else. But this is a technical issue tracker, so please make sure to focus on the tech and keep your opinions to yourself.

Thank you for reading through this primer. I look forward to working with you on this issue! Cheers :blue_heart:

[balthanon]

The original question, with slightly more detail on the issue can be found here: https://github.com/floccusaddon/floccus/discussions/1847

[balthanon]

I've tried a couple different ways to upload the redacted logs without success (individual files, multiple, a zip file, a 7-zip archive. All of them failed.) At this point I'm assuming it might be a permission issue, so let me know if the logs would be helpful and how to get them to you.

[balthanon]

A note on settings that might be relevant; I am currently using 2 or 3 profiles, depending on browser.

Firefox based browsers use a bookmark toolbar, bookmark menu, and other bookmarks (not used in Chromium) profiles.

Chromium browsers are typically bookmark toolbar and bookmark menu profiles; associated to the bookmark bar and other bookmarks folders typically, though it can vary slightly.

Nested folders are enabled for all profiles.

I will leave the issue active on my browsers for the time being, but will probably revert to a backup in the next couple days.

[marcelklehr]

Hey @balthanon I'm sorry for the trouble, I hope we can get to the bottom of this together. The logs are indeed invaluable for this. You could send them to me via email ( [email protected] ) or by sharing via Google drive.

[balthanon]

Hi, here's a Google Drive link to the redacted files. If you need the unredacted ones, let me know and we can work that out over email.

https://drive.google.com/file/d/1Kaq242JcYWxRZQ0W0ChThW9RKzwjgSfr/view?usp=sharing

[marcelklehr]

I think the logs you sent are from subsequent sync runs, sadly, not from the offending sync run that duplicated the folders. Perhaps the unredacted logs are from the offending sync run. The problem is that floccus only keeps the logs from the last sync run as browsers don't allow more space usage for storing logs of multiple runs :/ Also, could you outline your sync profile setup? ie. which folder on which browser will get synced to which file? Are there any nested folders that are synced with different profiles?

[balthanon]

I was afraid of that; it didn't look like it had much detail at all. The unredacted logs are pretty similar looking generally aside from the waterfox one-- I'll see about getting that to you via email.

Edit: Actually, I take it back-- it was the redacted waterfox log that was so much longer than the others.

Would it help if I opened a computer and browser that I haven't used since the issue occurred and pulled the log immediately after it adds the bookmarks? It presumably wouldn't capture the original issue, but I'm not sure if it would tell you something.

For my setup, I have the following:

3 Profiles: Bookmarks Menu, Bookmarks Toolbar, Other Bookmarks

Firefox (Bookmark Location > Profile):

• Bookmarks Toolbar > Bookmarks Toolbar
• Bookmarks Menu > Bookmarks Menu (this is the only one that seems to be impacted and it is where probably 2680 of my 2700 bookmarks reside)
• Other Bookmarks > Other Bookmarks

Waterfox

• Bookmarks Toolbar > Bookmarks Toolbar
• Bookmarks Menu > Bookmarks Menu
• Other Bookmarks > Other Bookmarks

Chrome

• Bookmarks Bar > Bookmarks Toolbar
• Other Bookmarks > Bookmarks Menu

Vivaldi

• Bookmarks Toolbar > Bookmarks Toolbar
• Bookmarks Menu > Bookmarks Menu
• Other Bookmarks > Other Bookmarks

As far as I can tell, none of the bookmark folders are really nested. They're all root level folders, unless there's something going on in the background that isn't really displayed in the bookmark managers. I thought I recalled something like that with Vivaldi that caused a few issues originally, but I'm not seeing any issues logged by me other than this one.

There are other browsers as well, including Opera, Edge, and Brave that are fully set up, but as far as I know they weren't open at the time. I installed Zen and Floorp a little while back and , but I haven't actually gotten the profiles set up on them yet.

[balthanon]

I assume the logs all share the same space? I've noticed as I'm reviewing them that despite being in the bookmarks menu profile, the logs are showing the Other Bookmarks and Bookmark Toolbar profile actions, so I don't think they're going to be any use either.

I wonder-- would it be possible to set it up so that logs are pushed to Google Drive instead of stored in the browser? That seems like it would provide more options than the browser storage provides. I assume it wouldn't necessarily work as well (or easily) on Nextcloud and some of the other options that don't use a cloud storage solution though.

I've been intending to get into doing some open source development; might be worth trying to work on some of the features that I've suggested at some point and the log storage seems like another potential feature request to add.

[marcelklehr]

Would it help if I opened a computer and browser that I haven't used since the issue occurred and pulled the log immediately after it adds the bookmarks?

That would not really help, no.

I've been intending to get into doing some open source development; might be worth trying to work on some of the features that I've suggested at some point and the log storage seems like another potential feature request to add.

That would be awesome. The log storage is a real problem, as floccus usually syncs every few minutes and then overrides crucial logs with uninteresting ones.

For now I've whipped up https://github.com/floccusaddon/floccus/pull/1902 which adds the addition failsafe as suggested by you in #1900. It also makes floccus apply the failsafe on upstream changes already instead of only local changes in order to catch the offending changes on the machine that they happened on, instead of on all other machines.

[marcelklehr]

IndexedDB may be an alternative to extension storage for logs as it seems to have more space (~50MB). I'm wary of using Google Drive because most users of floccus do not use Google Drive, I believe. There could be fallbacks of course to Nextcloud or git, but it's patchwork. Ideally in my opinion the extension should hold the logs itself.

[balthanon]

Very nice. For now, I think I'll go ahead and reapply my backups then, since it sounds like there isn't really anything we're going to get out of my machine by leaving it as is. If it happens again, ideally the failsafe will catch it and prevent additional changes, so even if the log changes don't happen prior to then, those will be the latest logs anyway.

I do agree it would be better to have the logs in the extension as well. For text 50 mbs seems like plenty too; particularly if you prioritize the ones you keep as only the latest and those where something actually changed. The vast majority of the logs are essentially going to show nothing happening.

[marcelklehr]

those where something actually changed

That's a nice idea. We can discard logs of sync runs where nothing was changed.

[wtto00]

Hello, is there any progress on this issue? I am using Git for synchronization, and I frequently encounter duplicate bookmarks. This problem occurs not only with newly created directories but also with individual bookmarks.

[marcelklehr]

Hi @wtto00 There has been no progress on this issue. Are you syncing bookmarks via other means in parallel to floccus?

[wtto00]

The synchronization method I used before was WebDAV, and I didn’t encounter such issues with that. It was only after WebDAV stopped working that I switched to using Git for synchronization.

[marcelklehr]

Are you syncing bookmarks via other means in parallel to floccus?

I was thinking maybe you're using something like Firefox Sync in parallel

[marcelklehr]

It was only after WebDAV stopped working that I switched to using Git for synchronization.

Why did it stop working? what was the issue?

[wtto00]

I was thinking maybe you're using something like Firefox Sync in parallel

I think I now understand what’s going on; it’s probably due to the synchronization issue with my Microsoft account that’s connected to Edge. I used it on two Windows computers.

Why did it stop working? what was the issue?

It seems that there’s always a mysterious lock file that appears and cannot be automatically deleted, which prevents synchronization from taking place. In order to restore normal functionality, the lock file must be manually removed. However, after a while, for some unknown reason, the same problem reoccurs.

[marcelklehr]

the lock file must be manually removed

Ah, I see. That can happen when you shut down your computer while floccus is syncing. But, nowadays there's a force sync button that you can click and the lock file should be removed automatically after half an hour to and hour.

[marcelklehr]

it’s probably due to the synchronization issue with my Microsoft account that’s connected to Edge.

That's a possibility. I really discourage setups where you sync bookmarks with browser-builtin services in addition to floccus.