Handling of quota exceeded errors in workbox-precaching

[jeffposnick]

trafficstars

Library Affected: workbox-precaching

This is closely related to https://github.com/GoogleChrome/workbox/issues/1308, but the exact analog is https://github.com/GoogleChromeLabs/sw-precache/issues/345

Let's leave runtime caching aside for this issue, and assume that there's a web app instance that has revision N of their assets precached. The web app is updated and deployed, and now there's an attempt to download and cache a few of the new/updated assets that make up revision N+1. workbox-precaching does this in the install handler, and will fail installation if any of the individual cache.put() operations fail. (Since all the assets from revision N are still cached at this point, it's not too hard to imagine scenarios in which caching the new assets brings you over quota.)

From the point of view of the web page clients, there are events fired on the associated service worker registration showing the service worker progressing from the installing state to the redundant state. Unless (somehow) quota is freed up at some point, that installing -> redundant transition will happen indefinitely, leaving the user stuck revision N of all the precached assets indefinitely.

We should do something to help developers out here. What about a pattern like:

if ('serviceWorker' in navigator) {
  window.addEventListener('load', function() {
    navigator.serviceWorker.register('/sw.js').then(function(registration) {
      registration.addEventListener('updatefound', function() {
        var newWorker = registration.installing;
        newWorker.addEventListener('statechange', function() {
          if (newWorker.state === 'redundant') {
            if ('storage' in navigator && 'estimate' in navigator.storage) {
              navigator.storage.estimate().then(function(results) {
                var percentUsed = (results.usage / results.quota) * 100;
                // Let's assume that if we're using 95% of our quota, then this failure
                // was due to quota exceeded errors.
                // TODO: Hardcoding a threshold stinks.
                if (percentUsed >= 0.95) {
                  // Get rid of the existing SW so that we're not stuck
                  // with the previously cached content.
                  registration.unregister();

                  // Let's assume that we have some way of doing this without inadvertantly
                  // blowing away storage being used on the origin by something other than
                  // our service worker.
                  // I don't think that the Clear-Site-Data: header helps here, unfortunately.
                  deleteAllCachesAndIBD();
                }
              });
            } else {
              // What about browsers that don't support navigator.storage.estimate()?
              // There's no way of guessing why the service worker is redundant.
            }
          }
        });
      });
    });
  });
}

and then encapsulate all that logic in the hypothetical new module described in https://github.com/GoogleChrome/workbox/issues/1129?

[jeffposnick]

Thinking out load some more: we might be able to address this inside of the workbox-precaching install handler, by detecting cache.put() quota errors there, and using it as a signal to unregister the current service worker and clear all caches.

That would have the benefit of not requiring developers to change the way they register their service workers, and we would more context about which caches/IDB entries need to be deleted if the cleanup happened inside of our service worker.

[gauntface]

1. (Self Service) We could clear the runtime cache and retry. If still fails, clear cache and unregister the service worker. This would fallback to the network and sw would start failing on install as normal.
2. (Dev Option) We offer a cb function and on failing precache, developer can do anything they want (including message page - clear cache - clear something else, .....)
3. Default to self service and dev cb results in replacing default behaviour

[raejin]

@jeffposnick In the case of Safari 11.1, it doesn't support the navigator.storage API atm. What are some of the other possibilities that will fall in the newWorker.state === 'redundant' statement being true?

I'm trying to tackle this problem in general regardless of the browser. The plan is to delete all the known caches generated by our service worker.

[tgangso]

Would like to see a solution for this!

[raejin]

@tgangso Relevant pull request for a solution https://github.com/GoogleChrome/workbox/pull/1505

[tgangso]

Very interesting, thanks @raejin

[srosset81]

@jeffposnick Since we've updated from Workbox 2 to 3, we have Quota exceeded errors when the SW is updating, which makes the update sometimes fail. I've used the purgeOnQuotaError option on all runtime caching but it doesn't solve the problem, as indeed you write in #1505 :

If you're precaching a massive amount of data, but not runtime caching much, this PR is not likely to help.

What are the solutions ? In v2 it worked fine, maybe because there was no -temp cache during the SW installation ?

[jeffposnick]

There's not a specific solution in place for Workbox v3.

You'd have to trim down the amount/size of URLs that you're precaching.

Coming up with an automated solution is going to require more thought, and is not something that we've been able to address yet.

[budarin]

@jeffposnick is there any solution to this issue?

I am currently using the code:

navigator.serviceWorker
.register(swPath})
.then((registration) => {
    registration.addEventListener('updatefound', () => {
        const installing = registration.installing;
    
        if (installing) {
            const onInstallingStateChange = () => {
                if (installing.state === 'redundant') {
                    // redirect
                    // or alert about quote error
                }
    
                if (installing.state === 'installed') {
                    installing.removeEventListener('statechange', onInstallingStateChange);
                    resolve(registration);
                }
            };
    
            installing.addEventListener('statechange', onInstallingStateChange);
        }
    });
})
.then((registration) => {
   // start app
})

[jeffposnick]

Hello @budarin—there's no one-size-fits all solution here, so we've kept this open to track possibilities.

Listening for an installing service worker entering the redundant stage is a valid way of detecting it from the main page.

Alternatively, if you wanted to perform some asynchronous action from inside your your service worker when there's a quota error, you can use registerQuotaErrorCallback() from workbox-core to automatically run your code. This function could potentially purge some runtime caches, clear out IndexedDB, or even call self.registration.unregister(), depending on what makes the most sense for your use case.

In the future, we may expose a cacheWriteDidFail strategy lifecycle event (#1530), allowing you to add a custom plugin to the strategy used by workbox-precaching, and take specific action when a given cache write fails.

[budarin]

Hi @jeffposnick !

Is there any workbox precache method or callback params method to clear pre cache in case of quota exceeded error?

[jeffposnick]

Clearing the cache can be done with the standard Cache Storage API methods:

import {cacheNames} from 'workbox-core';

// Call this as your custom quota error callback, or anywhere else:
async function deletePrecache() {
  await caches.delete(cacheNames.precache);
}

Note that the next time the service worker installs, it's going to retry caching all the items in the precache manifest. So you might just end up redownloading everything and then running into quota issues again, unless the user also frees up additional space on the device or unless you reduce the size of your precache manifest.