web-llm

web-llm copied to clipboard

It would be great to have Gemma 7B support on top of Gemma 2B.

beaufortfrancois

[Announcement] Breaking changes regarding conversation template

1

Hi community, we recently updated our models' `mlc-chat-config.json` on hugging face to use the latest conversation template. The goal is to make templates more lightweight, hence not requiring an npm...

CharlieFRuan

sohsohsiying

Cannot find module 'tvmjs' or its corresponding type declarations EVEN has ‘npm install tvmjs’ done

12

(gh_web-llm) amd00@asus00:~/llm_dev/web-llm$ npm run build > @mlc-ai/[email protected] build > rollup -c src/index.ts → lib/index.js... [!] (plugin rpt2) Error: src/chat_module.ts:1:24 - error TS2307: Cannot find module 'tvmjs' or its corresponding type...

SeekPoint

[Tracking] WebLLM: OpenAI-Compatible APIs in ChatModule

1

## Overview The goal of this task is to implement APIs that are [OpenAI API](https://platform.openai.com/docs/api-reference) compatible. Existing APIs like `generate()` will still be kept. Essentially we want JSON-in and JSON-out,...

CharlieFRuan

simple-chat: error during loading params onto WebGPU, GPUPipelineError: A valid external Instance reference no longer exists

3


lebron8dong

can I run in nodejs?

6

how can I make get-started run in nodejs?

mtiger2k

try to run gemma-7b but failed

1

Recently, I integrated webllm into my web project, and the effect of gemma-2b is pretty good. Thanks for your work, everything runs very well. I am trying to add more...

mayneyao

[Bug] If the download of the weighs fails it does not recover

6

If the download fails the model can not recover and will fail always unless the cache is cleaned. related to #280 related to #284

DavidGOrtega

I am seeing 100% RAM usage in my laptop when running this. Can you please let me know if its an issue or the minimum RAM requirement is higher than 8 GB?

1


devashish234073