web-llm
web-llm copied to clipboard
mlc-ai
High-performance In-browser LLM Inference Engine
Hi there **Web-LLM looks very promising but a few some key questions are missing for a newcomer to get the pick picture understanding in order to pick up the details...
chrome extension not working with manifest version 3 and Google will be stopping manifest version 2 in January, 2024 Are there any plans to make the extension work with manifest...
Install and run the current simple-chat example, I used both node v18 and v20. While it works with the llama-2 models, using the Redpajama or Vicuna models will generate this...
Callback is a very outdated API, would it be possible to use modern javascript? ```ts import * as webllm from "@mlc-ai/web-llm"; for await (const response of webllm.chat.completions.create({ model: 'mistral-7b-chat-hf-q4f32_1', stream:...
Hi! 👋 Firstly, thanks for your work on this project! 🙂 Today I used [patch-package](https://github.com/ds300/patch-package) to patch `@mlc-ai/[email protected]` for the project I'm working on. I use the context_free setting and...
Is restoring context / previous messages / responses supported? If so, how would one go about this? I noticed on the worker API there is a `ChatModule.prefill` method but I'm...
### Issue Hi there, I'm trying to run `simple-chat` and make changes to `web-llm` locally while running it. I'm following [these instructions](https://github.com/mlc-ai/web-llm/blob/main/README.md#build-webllm-package-from-source) to build from source but getting: ``` [!]...
Hello. Everything seems to be set up, but no response. Any suggestions?   
