Handling Firefox quota enforcement

[twolfson]

Type: code issue

For the past couple weeks, I've been able to disable sites for a session but upon browser restart, the disabled sites were no longer registered

Steps:

• Open a link in an email
• Get prompted by HTTPS Everywhere to handle non-HTTPS site
• Select "Open insecure page" (permanent)
• Go to website without issue
• Close browser
• Open same link in email

Expected behavior:

• No prompt from HTTPS Everywhere

Actual behavior:

• Prompted again by HTTPS Everywhere to handle non-HTTPS site

Research:

• Dug into developer tools for HTTPS Everywhere
• Noticed "QuotaExceededError: storage.sync API call exceeded its quota limitations." on site add/deletion
• Found out Firefox added storage.quota.sync enforcement in Firefox 79, https://blog.mozilla.org/addons/2020/07/09/changes-to-storage-sync-in-firefox-79/comment-page-1/
• Quota for single item is: 8192 bytes, https://searchfox.org/mozilla-central/rev/8d55e18875b89cdf2a22a7cba60dc40999c18356/toolkit/components/extensions/schemas/storage.json#314-325
• Played with chrome.storage.sync.set({disabledList: [...]}) in HTTPS Everywhere to see when it stopped spitting out errors
  • Old data (got error): 14412 bytes
  • New data (no error): 7568 bytes (prob could run up to 8192 but conclusive enough)

Discussion:

• Solutions to work around 8192 byte limitation will eventually run into same issues (i.e. total storage is 102400 bytes, gzipping saves maybe 50% of data)
• Probably better to check usage via getBytesInUse then notify user they need to remove existing sites
• Maybe Mozilla can be convinced to increase their quota (unsure if same issue exists in Chrome)
• I'm sure y'all have better ideas than me around this =)

Workaround for now:

• Open "Add-Ons"
• Navigate to "HTTPS Everywhere" extension
• Click on gear then "Debug Add-ons"
• Clear existing disabled sites via: chrome.storage.sync.set({'disabledList': []}, console.log)
• Restart browser
• Wait for another couple months to run into limit
• Rinse and repeat

[pipboy96]

@zoracon Maybe we can move towards using local browser storage instead of the sync one, or allow the user to choose? As far as I know, local storage is almost unlimited in its size.

[twolfson]

A few lines down from chrome.storage.sync settings is chrome.storage.local. The limitation there is 5MB instead of 100KB total, definitely more reasonable:

https://searchfox.org/mozilla-central/rev/8d55e18875b89cdf2a22a7cba60dc40999c18356/toolkit/components/extensions/schemas/storage.json#341-351