web-llm
web-llm copied to clipboard
Use modern javascript API
Callback is a very outdated API, would it be possible to use modern javascript?
import * as webllm from "@mlc-ai/web-llm";
for await (const response of webllm.chat.completions.create({
model: 'mistral-7b-chat-hf-q4f32_1',
stream: true,
messages,
})) {
console.log(res)
// You, can, use, ...
}
ideally it would be similar UX that the client side of
https://github.com/vercel/ai
but i guess not everyone use react