web-llm icon indicating copy to clipboard operation
web-llm copied to clipboard

Use modern javascript API

Open louis030195 opened this issue 2 years ago • 0 comments

Callback is a very outdated API, would it be possible to use modern javascript?

import * as webllm from "@mlc-ai/web-llm";

for await (const response of webllm.chat.completions.create({
    model: 'mistral-7b-chat-hf-q4f32_1',
    stream: true,
    messages,
})) {
  console.log(res)
  // You, can, use, ...
}

ideally it would be similar UX that the client side of

https://github.com/vercel/ai

but i guess not everyone use react

louis030195 avatar Oct 08 '23 23:10 louis030195