multi-account-containers icon indicating copy to clipboard operation
multi-account-containers copied to clipboard

Published 20 hours ago • verified publisher icon mozilla

Sync Containers with Temporary Account Containers

Open ghost opened this issue 4 years ago • 31 comments

Information

  • Multi-Account Containers Version: 6.2.3
  • Firefox Version: 73.0.1
  • Other installed Add-ons + Version + Enabled/Disabled-Status: Temporary Containers v1.8

Actual behavior and Expected behavior

It is syncing all Container Labels and works fine

Problem

It is also syncing all Containers from the Temporary Account Containers Addon. Temporary Account Container is deleting the tmp containers after the Tab is closed for 2 minutes. So it is often synced to the cloud. So i am getting on all other Firefox instances the "tmp**" labels from my other clients. It would be good if it is possible for example to disable syncing all containers with Syntax "tmp**"

Feature Request

Allow to disable/ignore syncing for cointainers with special characters or syntax

Steps to reproduce

  1. Use Multi Account Container with temporary Containers
  2. Create a new TMP Container
  3. It will be Synced with Firefox Sync

ghost avatar Mar 06 '20 10:03 ghost

I have the same problem and it get so serious that I am unable to synchronize anything at all due to the huge number of temporary containers synchronized


1584168207630	Sync.Engine.Extension-Storage	ERROR	Syncing @testpilot-containers: request failed: Error: HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Maximum bytes per object exceeded " "(39015 > 16384 Bytes.)(resource://services-common/kinto-http-client.js:2771:5) JS Stack trace: [email protected]:2771:5
[email protected]:2925:13
1584168207630	Sync.Engine.Extension-Storage	WARN	Syncing failed: Error: HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Maximum bytes per object exceeded " "(39015 > 16384 Bytes.)(resource://services-common/kinto-http-client.js:2771:5) JS Stack trace: [email protected]:2771:5
[email protected]:2925:13

I have also tried to manually delete the temp containers from ~/.mozilla/firefox/PROFILENAME/containers.json; at the startup the containers are not shown, but they gradually start to pop up again after a while.

EDIT: Obviously the error also tells that now the main problem is how to delete the remote list of containers, since it appears that once you exceed the quota you are unable not only to synchronize, but also to delete stuff.

tigerjack avatar Mar 14 '20 06:03 tigerjack

Also going to agree. Container sync is unusable with temp container usage. I have at least hundreds of synced tmp*, and dozens of the same tmp# value. It's not possible to remove them at this point, and the only way to fix it is to completely delete all my sync data for the entire browser. PLEASE add wildcard sync exclusions.

alongfield avatar Mar 18 '20 13:03 alongfield

Same here. Hundreds of containers. Impossible to remove, they keep coming back from sync. I have not found out any workaround.

wasowski avatar Mar 28 '20 11:03 wasowski

I had this issue recently too but with a worse side-effect.

With sync "Add-ons" options enabled in Preferences I kept getting similar errors:

Sync.ErrorHandler DEBUG extension-storage failed: Error: HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Maximum bytes per object exceeded " "(27087 > 16384 Bytes.)(resource://services-common/kinto-http-client.js:2771:5) JS Stack trace: [email protected]:2771:5

While erroring out Firefox kept writing these error logs to the profile directory: /home/<user>/.mozilla/firefox/<profile>.default/weave/logs/

After each error Firefox tried to sync the profile again instantly (or few seconds delay).

And that'd be fine, except it kept happening non-stop and silently, no error notification whatsoever.

Problem: I noticed in the past weeks that my whole system performance was subpar but didn't have time to track it down and finally did so yesterday.

Investigation: The problem was I/O bound, Firefox had filled 17 GB worth of text error logs at: /home/<user>/.mozilla/firefox/<profile>.default/weave/logs/

The constant I/O caused by Firefox hurt the whole system performance and kept crashing Firefox itself. (I've sent more than 10 crash reports recently with the built-in system.)

The constant I/O problem was caused by Firefox writing sync errors logs with Insufficient Storage reason for extension-storage.

The sync errors were also constant as a new sync attempt happened after each failure.

The sync errors happened while having Add-ons option enabled on Sync Settings.

Expected results:

  • Syncing Add-ons should have worked.
  • In case of repeated errors of the same kind the process should not continue indefinitely.
  • If the process is enabled and cannot be completed successfully the user should be notified.

Observed results:

  • Syncing Add-ons always failed.
  • The syncing process kept restarting indefinitely.
  • There was no explicit error feedback of any kind.
  • Error logs were created for each attempt.

Workaround:

  • Disable Add-ons sync setting.

Usage notes: I mainly use Firefox (Nightly) on Linux and rarely launch the browser in my gaming Windows virtual machine.

I use both Multi-Account Containers and Temporary Containers add-ons.

I never really pay attention to sync details and just expected it to work silently.

As a browser (and add-ons) I don't expect it to be required to manually check the presence of error logs to identify non-recoverable issues of a set-and-forget feature.

System:

Application Basics

Name 		Firefox
Version 	76.0a1
Build ID 	20200327215207
Update Channel 	nightly
OS 		Linux 5.5.13-arch1-1

aruraune avatar Mar 31 '20 15:03 aruraune

I'm seeing the error above even without temporary containers, and it has broken local storage in my profile. I've reported the issue upstream at https://bugzilla.mozilla.org/show_bug.cgi?id=1624586.

jonhoo avatar Mar 31 '20 15:03 jonhoo

Just checked my weave directory and... what the hell, ~1000 files in 9 days and 2 GiB of logs!

tigerjack avatar Apr 01 '20 12:04 tigerjack

Found these log files as well here. I had opened #1699 earlier, but I'll close it in favor of this one.

mathstuf avatar Apr 02 '20 22:04 mathstuf

Hmm. Disabling sync in the extension also doesn't seem to avoid the error.

mathstuf avatar Apr 02 '20 22:04 mathstuf

In the meantime, I've added the log directory to my backup's exclusion (my directory was ~16GB before nuking it).

I'll note that it doesn't seem like anything else is not being synced properly at least, so it's not completely useless. But a way to poke into what is being synced and manually delete whatever is taking up too much space would be great.

mathstuf avatar Apr 02 '20 22:04 mathstuf

I too have this issue. For some reason the number of temporary containers grew to a very large number without me noticing it and now I get sync error on every sync. My logs took up 9GB. Doesn't help to turn off container syncing, I still get the sync error.

erikellsinger avatar Apr 05 '20 17:04 erikellsinger

Quick fix/what helps I configured my temporary containers to reuse the old tmp numbers. So it helps it isnt creating endless containers and syncs the same numbers again.

Problem still persists but it doesnt the break my syncing completly

ghost avatar Apr 05 '20 18:04 ghost

I don't think this is just related to temporary account containers. I do not have that extension installed, yet enabling sync exceeds the per-extension sync quota for me. Specifically, I end up with entries in my sync log like:

Syncing @testpilot-containers: request failed: Error: HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Collection maximum size exceeded (215842 > 204800 Bytes).)

This makes me think that either too much stuff is being synced, or multi-account container configs are large enough that the 200kB limit is too small.

jonhoo avatar Apr 07 '20 15:04 jonhoo

As a workaround for now, is there a way to remove all remotely held sync data for Firefox. We can then start over and disable the MAC syncing until this gets resolved?

Right now, I cannot use any syncing and firefox generates around 12GB of error logs every other day. I would at least like a way to clear out the remote storage (perhaps taking the current browser instance as the new source of truth and syncing from there).

I should add, I mean remove just the addon syncing, I dont want to lose everything else in my firefox account.

Poggles avatar Apr 15 '20 13:04 Poggles

If you're getting the "Insufficient Storage" error, a workaround that might help for now is disabling MAC sync and clearing its sync storage like so:

  • Backup your profile(s)
  • Disable the MAC sync feature on all devices
  • Navigate to about:debugging#/runtime/this-firefox
  • Click "Inspect" right to Multi-Account Containers
  • Click "Console"
  • Insert into the console
await browser.storage.sync.clear();
console.log("done");
  • If you're getting a "Scam warning", type "allow pasting" into the console, remove it again, and then insert code from last step again
  • Press Enter and wait for the console to print done

stoically avatar Apr 18 '20 08:04 stoically

Thanks for this, but sadly it doesn't work. After following the above, i get undefined in the response to the await. If I disable the addons and then try it, then it seems to work (hard to tell), and I no longer get sync errors. However, once the addon is re-activated i get the errors again.

Interestingly, it seems there are two other add-ons failing in the logs. I don't know if that's because those add-ons also cause issues, or if its just because the store is full, so all syncing add-ons will fail.

I guess I could play with this a little, by creating a fresh browser instance in a container and seeing if the sync storage really was wiped or not. I dont have time for this currently, but will take alook later in next week.

Thanks again.

Poggles avatar Apr 18 '20 09:04 Poggles

After following the above, i get undefined in the response to the await.

The undefined is expected - but if it never prints done, then that hints to something being seriously broken with the sync storage on Firefox's side. Do you see any errors when executing the code snippet?

stoically avatar Apr 18 '20 12:04 stoically

I do get the console "done" response, so it seems to be working. I do not get any errors when the snippet executes, but there are a lot of other, unrelated, errors being generated when I leave that screen open from other extensions.

Poggles avatar Apr 18 '20 13:04 Poggles

@stoically I can confirm that running await browser.storage.sync.clear(); does not have an effect on relieving the logging of that error

await browser.storage.sync.get()
{…}
"MACinstance:f144d15a-4890-4b1a-a1e4-82582815cff5": Object { timestamp: 1587928531113, identities: (6) […], siteAssignments: (12) […] }
deletedIdentityList: Array(32) [ "f2725761-f7b7-44a5-8ba5-2c458c2b7aa0", "ee2ce479-7a67-488e-ba48-28a9eef37f9a", "ea698591-5131-459d-8b7b-d9e64f712e06", … ]
deletedSiteList: Array(3) [ "siteContainerMap@@_<redacted>", "siteContainerMap@@_<redacted>", "siteContainerMap@@_<redacted>" ]
​"identity@@_0a2f8e8e-d518-469e-8212-eb3f662dc4ef": Object { name: "3", icon: "circle", color: "red", … }
"identity@@_1bbc2cb9-c41f-4349-9063-5085546fda75": Object { name: "15", icon: "circle", color: "red", … }
​"identity@@_39e8b21b-5068-40c6-a534-94b32764b82a": Object { name: "13", icon: "circle", color: "red", … }
​"identity@@_3de9c834-24c3-4a1b-94d9-f4fd694ee375": Object { name: "Banking", icon: "dollar", color: "green", … }
​"identity@@_46b3d751-3624-4389-8811-0607357023d6": Object { name: "12", icon: "circle", color: "red", … }
"identity@@_54eefbe6-06d2-4342-a27a-75ad38a8380d": Object { name: "29", icon: "circle", color: "red", … }
​"identity@@_551227c7-2b14-4cbe-a1b1-1b068fc14f79": Object { name: "Google", icon: "tree", color: "green", … }
​"identity@@_62433ea0-649a-4d05-b68f-ab14e8ad99c1": Object { name: "aws", icon: "circle", color: "blue", … }
​"identity@@_64af82b2-584c-4dde-9879-9596b88cf5a7": Object { name: "2", icon: "circle", color: "red", … }
​"identity@@_9372ae84-3a3e-4df2-8428-1e0aae7b11d0": Object { name: "26", icon: "circle", color: "red", … }
​"identity@@_997d60e7-2958-4083-9c0e-7b473dfdaf6b": Object { name: "34", icon: "circle", color: "red", … }
​"identity@@_9fa843f0-f978-4157-b2d2-1e240e820c03": Object { name: "1", icon: "circle", color: "red", … }
​"identity@@_b2c53132-dd23-4e53-b7ee-0bf23803e273": Object { name: "Shopping", icon: "cart", color: "pink", … }
​"identity@@_bef0c939-2653-4a3f-b28e-9b8a41ac4637": Object { name: "Personal", icon: "fingerprint", color: "blue", … }
​"identity@@_bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5": Object { name: "<redacted>", icon: "fingerprint", color: "blue", … }
​"identity@@_c09f7fae-f2b2-4e12-8fb3-bb8cbdb6bc55": Object { name: "24", icon: "circle", color: "red", … }
​"identity@@_d9ef128e-f2eb-4552-8df4-f2befac5a1e8": Object { name: "Amazon", icon: "cart", color: "green", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: false, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: true, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: false, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: true, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: false, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "12", neverAsk: false, identityMacAddonUUID: "d9ef128e-f2eb-4552-8df4-f2befac5a1e8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: false, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "7", neverAsk: false, identityMacAddonUUID: "fc4909fa-18ce-4c69-b826-12a123d1f5b8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "54", neverAsk: true, identityMacAddonUUID: "26899cff-1b2b-4937-bd6c-ebc7301979f8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: false, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​<prototype>: Object { … }
await browser.storage.sync.clear()
undefined
await browser.storage.sync.get()
{…}
"MACinstance:f144d15a-4890-4b1a-a1e4-82582815cff5": Object { timestamp: 1587928787665, identities: (6) […], siteAssignments: (12) […] }
​"identity@@_3de9c834-24c3-4a1b-94d9-f4fd694ee375": Object { name: "Banking", icon: "dollar", color: "green", … }
​"identity@@_551227c7-2b14-4cbe-a1b1-1b068fc14f79": Object { name: "Google", icon: "tree", color: "green", … }
​"identity@@_b2c53132-dd23-4e53-b7ee-0bf23803e273": Object { name: "Shopping", icon: "cart", color: "pink", … }
​"identity@@_bef0c939-2653-4a3f-b28e-9b8a41ac4637": Object { name: "Personal", icon: "fingerprint", color: "blue", … }
​"identity@@_bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5": Object { name: "<redacted>", icon: "fingerprint", color: "blue", … }
​"identity@@_d9ef128e-f2eb-4552-8df4-f2befac5a1e8": Object { name: "Amazon", icon: "cart", color: "green", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: false, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: true, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: false, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: true, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "19", neverAsk: false, identityMacAddonUUID: "bf9c2c3a-790f-4c92-8417-ccd4de4ca3b5", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "12", neverAsk: false, identityMacAddonUUID: "d9ef128e-f2eb-4552-8df4-f2befac5a1e8", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: true, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​"siteContainerMap@@_<redacted>": Object { userContextId: "53", neverAsk: false, identityMacAddonUUID: "bef0c939-2653-4a3f-b28e-9b8a41ac4637", … }
​<prototype>: Object { … }
Note: I redacted some details using <redacted>

Sync logs before

HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Collection maximum size exceeded (206535 > 204800 Bytes).)(resource://services-common/kinto-http-client.js:2771:5) JS Stack trace: [email protected]:2771:5

Sync logs after

HTTP 507 Insufficient Storage: Resource access is forbidden for this user (Collection maximum size exceeded (206535 > 204800 Bytes).)(resource://services-common/kinto-http-client.js:2771:5) JS Stack trace: [email protected]:2771:5

Everything done with container sync turned off with FF 75.0 and a fresh profile sync'ing to the account with the issue.

casual-will avatar Apr 26 '20 19:04 casual-will

I found this issue from awhile ago https://github.com/mozilla/notes/issues/791#issuecomment-375708159 . Is it possible once we hit the limit (trigger the 507) we are unable to recover space from remote objects on the server? At this point I think I've confirmed locally I don't have that many container sync objects.

casual-will avatar Apr 26 '20 19:04 casual-will

It seems that the log deluge no longer occurs, though I'm unsure if it is a Firefox, add-on, or sync server change that fixed it.

I am seeing noticeable stalls when syncing now because it is trying to sync 100s of temporary containers down. I try to delete them locally, but they sync back up from the server again. Can we please have an option for masking out containers of a given name from syncing?

mathstuf avatar Jun 24 '20 15:06 mathstuf

Is there any way to recover a Firefox account from the "507 Insufficient Storage" issue?

I still can't use MAC with sync ever since it got flooded by temporary containers. Enabling sync pulls back in around 4000 temporary containers to my local Firefox from the sync server. Deleting them locally (using https://github.com/stoically/temporary-containers/issues/371#issuecomment-615861439) again doesn't get synced with the server (because of 507 error)

I see the similar Notes sync issue mentioned above was fixed in https://github.com/mozilla/notes/pull/824. Maybe something similar can be applied to MAC as well?

avian2 avatar Dec 15 '20 11:12 avian2

I've been trying to debug this 507 Insufficient Storage issue.

To do requests to the sync server directly I used the following in the browser console:

const { extensionStorageSyncKinto } = Cu.import("resource://gre/modules/ExtensionStorageSync.jsm", {});
const { KintoHttpClient } = Cu.import("resource://services-common/kinto-http-client.js", {});

await extensionStorageSyncKinto._requestWithToken("test", function(token) {
	const headers = { Authorization: "Bearer " + token };
	const kintoHttp = new KintoHttpClient("https://webextensions.settings.services.mozilla.com/v1", {
	        headers: headers,
	});
	return kintoHttp.[something - see below]
    });

kintoHttp methods are listed here. There's one bucket. You can get it with

kintoHttp.listBuckets()

The bucket has several collections named ext-.... I'm guessing one collection per extension, but I couldn't find a way of connecting the collection name with the extension.

kintoHttp.bucket(<bucket>).listCollections()

In my case, one of the collections had around 300 records and others were empty:

kintoHttp.bucket(<bucket>).collection(<collection>.listRecords()

I assumed the collection with 300 records was associated with the MAC extension. Deleting the collection for it results in the 507 error turning into:

Syncing @testpilot-containers: request failed: TypeError: NetworkError when attempting to fetch resource. No traceback available

Re-making an empty collection with the same name results in the 507 Insufficient Storage reappearing with the next sync and the collection is again full of records. This also happens if clearing the collection immediately after running await browser.storage.sync.clear(); However the error message now says Collection maximum size exceeded (205191 > 204800 Bytes. Before clearing out the collection it said Maximum bytes per object exceeded (124495 > 16384 Bytes.

about:preferences#containers only shows 5 containers, so locally I currently don't have an excessive number of containers defined.

This is as far as I got. I don't know why MAC still tries to store an excessive amount of data on the sync server even with a) local sync storage cleared out b) reasonable number of containers defined and c) remote data cleared manually. At least it seems that the source of the problem is not the server refusing to recover space after hitting the 507 error.

avian2 avatar Jan 27 '21 10:01 avian2

I can testify from some experiment I've done for the last 3 days about deleting, Creating new account (Firefox account), Deleted every hosts, Readded it, Wait some times,

The sync log are not cleared about it.

But the add-on temporary containers disable the ability to sync apparently. So is it wanted like that by the add-on? By the sync process of Firefox itself? Something else?

changemenemo avatar Oct 14 '21 09:10 changemenemo

I can also confirm this issue still persists for me with the latest 7.4.0 update. I enable sync, get all my containers, delete them, account sync: Get loads of Uncaught (in promise) Error: QuotaExceededError: storage.sync API call exceeded its quota limitations. just like before.

Was the unlimitedStorage permission not intended to resolve this issue?

domohawk avatar Oct 23 '21 04:10 domohawk

Hello,

The issue still not fixed since many year. And I see also that the PR was revered (8e7d9f7574fd4834b72d217fb3f63c697cd46b72). So @dannycolin why did you reverded this change ?

And is it possible to have one day a fix about this ?

Josue-T avatar Feb 16 '24 20:02 Josue-T

It was reverted via #2450 in case that answers your question.

mathstuf avatar Feb 17 '24 06:02 mathstuf

Thanks for the answer. I understand more why this revert.

So can you open again this issue as it still not be resolved now.

Josue-T avatar Feb 18 '24 07:02 Josue-T

I cannot reopen; I'm only a user like you :) .

mathstuf avatar Feb 19 '24 04:02 mathstuf

Maybe @dannycolin can open it again ?

Josue-T avatar Feb 22 '24 07:02 Josue-T

I can reopen it. However, the author of Temporary Containers passed away last year. Also as an unpaid contributor, I do not have any free time to work or review a patch for this bug. Finally, I can't push a new release on addons.mozilla.org because I don't have the access. So, we'd need someone at Mozilla for that.

dannycolin avatar Feb 22 '24 19:02 dannycolin