web-llm icon indicating copy to clipboard operation
web-llm copied to clipboard

IndexedDB cache fails like the caches

Open DavidGOrtega opened this issue 1 year ago • 7 comments

IndexedDB was added to avoid #257 However in the previously reported OS with Chrome browser the issue is similar. While the loading is completed at 100% the files are not there and load fails at the end.

Uncaught (in promise) Error: ArtifactIndexedDBCache failed to fetch: http://localhost:8082/dolphin-2_6-phi-2/MLC/params_shard_0.bin

The indexedDB cache only presents 4 out of 50 bin files

image

In chromium works as expected

image

DavidGOrtega avatar Apr 21 '24 18:04 DavidGOrtega

Its related to the cache quota. For some reason in this OS and Chrome is 345MB

image

DavidGOrtega avatar Apr 21 '24 19:04 DavidGOrtega

I can it force to be bigger and it works 🤔

image

DavidGOrtega avatar Apr 21 '24 19:04 DavidGOrtega

Interestingly enough if I open a new tab without any URL the inspector shows a much larger limit

image

DavidGOrtega avatar Apr 21 '24 20:04 DavidGOrtega

The problem is related on how Chrome manages quota and how much can be stored. The maximum data allowed to store is severely restricted depending on opaque responses, incognito sessions and a very common setting that seems to be found enabled in MacOS by default:

If the user has enabled "Clear cookies and site data when you close all windows" in Chrome, the storage quota is significantly reduced to a maximum of approximately 300MB.

To avoid the restriction and be able to have bigger cache just change this setting on Chrome. To my mind this is a bump. Also que can confirm that IndexedDB is far better that caches as the limit is much higher.

image

DavidGOrtega avatar Apr 21 '24 22:04 DavidGOrtega

I think this information would be very valuable in docs.

DavidGOrtega avatar Apr 22 '24 02:04 DavidGOrtega

Really appreciate the findings! Will update the doc

CharlieFRuan avatar Apr 22 '24 04:04 CharlieFRuan

For users who can't change the "Clear cookies and site data when you close all windows" settings in Chrome (e.g. when you use a company laptop with organisation-managed settings), the only option that seems to work is to pack code into a Chrome extension with unlimitedStorage permission, based on my tests (on M1 Mac & Chrome 125.0.6422.77 ).

e.g. the chrome-extension-webgpu-service-worker example, you will need to change the permissions in manifest.json to:

"permissions": ["unlimitedStorage", "tabs", "webNavigation"]

t83714 avatar Jun 15 '24 10:06 t83714