web-llm
web-llm copied to clipboard
mlc-ai
Invalidate cached WebAssembly binaries
The model-specific WebAssembly binaries are cached and stored in the client-side cache on the user’s device (webllm/wasm). However, the JavaScript portion of WebLLM is distributed with the application. This can lead to scenarios where the library and the Wasm are updated, but the user still has an old WebAssembly binary in their cache, breaking the application (see #221). I propose introducing an invalidation system (e.g., using versions or hashes, etc.). If the WebLLM JS library is not compatible with the Wasm binary stored locally, it should remove the cache entry and attempt to download the Wasm binary again.
Thanks for the suggestion! We are thinking about adding versioning to the binary name, say Llama-v0.2.21, then when updating it we update it to Llama-v0.2.22 so the outdated one in user's cache will not be used and be removed.
but what recently changed that a re download was needed of the model? we can't make each user download 10+GB too many times... thanks
I don't think the recent changes require redownloading. Besides, if we were to redownload a model, only the model library (tens of MB) is needed, not the weights.
it's been already a few times that even on the chat demo, I need to clear the cache to be able to to make it work. it just happens in the last release of the npm web llm, I updated from .20 to .23 and get this error, I can't find the reason, when calling reload mistral model get ExitStatus {name: 'ExitStatus', message: 'Program terminated with exit(1)', status: 1} when it is on 40-60%. if I clear the cache it works but then it can't load from cache again. if I go back to .20 it works ok. what it could be? thanks - using chrome on mac
it's been already a few times that even on the chat demo, I need to clear the cache to be able to to make it work. it just happens in the last release of the npm web llm, I updated from .20 to .23 and get this error, I can't find the reason, when calling reload mistral model get
ExitStatus {name: 'ExitStatus', message: 'Program terminated with exit(1)', status: 1}when it is on 40-60%. if I clear the cache it works but then it can't load from cache again. if I go back to .20 it works ok. what it could be? thanks - using chrome on mac
Same happens to me for every model on chrome extension. Do not know how to fix it. I am also using chrome on mac
@UXDart This shouldn't be related to the model library versioning issue described by the OP. Seems to be more related to https://github.com/mlc-ai/web-llm/issues/322 and https://github.com/mlc-ai/web-llm/issues/313. Will look into this issue today; meanwhile clearing the cache might alleviate the issue as a temporary workaround.
@UXDart @kudaibergenu Just updated the npm to 0.2.24: https://github.com/mlc-ai/web-llm/pull/323
This should fix it. Let us know if issues persist. Thank you!
Update: we now support wasm versioning such that whenever a new version of wasm is required, we will download the new wasm (since the URL is different). The idea is that each web-llm npm version binds to a wasm version. See the PR below for more:
- https://github.com/mlc-ai/web-llm/pull/355