Charlie Ruan
Charlie Ruan
OOM errors in `createBuffer()` are now able to be caught in webllm npm 0.2.36 via https://github.com/mlc-ai/web-llm/pull/402. If `createBuffer()` does not suffice, will follow up with more error catching in tvmjs....
Closing this issue as completed; feel free to open new ones if problems persist!
> I can definitely take this one! Im actually taking it for a project of mine. @DavidGOrtega Thanks for offering help! You are referring to item C3 right? > My...
I think we can go with your suggestion and use level here. Looking forward to the change!
Hi @germain-gg! I believe currently the downloads are resumable, as weights are broken into shards (e.g. ~105 shards for Llama-3-8B). For each shard that finishes downloading, it would be cached....
Really appreciate the findings! Will update the doc
This error should be addressed in npm 0.2.36. For details please see https://github.com/mlc-ai/web-llm/pull/397.
Function calling is a WIP and will be updated soon! Apologies for the inconvenience. All other examples should be working and feel free to try them out
Hi @mwyrzykowski, thanks for the support and sorry for the delayed response. WebLLM after npm 0.2.47 should fall back to `maxBufferSize` 256MB in case the 1024MB hits the limit: https://github.com/mlc-ai/web-llm/pull/498...
This weekend / early next week!