I downloaded the model on huggingface locally, can I load the local model?

[menghan465]

I downloaded the model on huggingface locally, can I load the local model?

[atom2-source]

Yes with extra work im doing that now.

[robertgro]

https://webllm.mlc.ai/docs/user/basic_usage.html#creating-an-mlcengine

[CharlieFRuan]

This example should help: https://github.com/mlc-ai/web-llm/tree/main/examples/simple-chat-upload

[cometta]

@CharlieFRuan Is there anyway to read .bin directly without first save in either indexdb or caches?

[CharlieFRuan]

@cometta Hmm is there a specific reason for this? We do have APIs to delete the model weights from cache

[cometta]

If the model binary files already exist locally, the HTTP server should be configured to read them directly from the local filesystem https://localhost/file.bin, rather than downloading them from a URL and saving them to the cache again

[mchaliadzinau]

This example should help: https://github.com/mlc-ai/web-llm/tree/main/examples/simple-chat-upload

How it should work exactly?

For instance I've downloaded gemma-2-9b-it-q4f32_1-MLC and started simple-chat-upload.

Is it expected that I'll select all files which I've downloaded or only .bin files?