web-llm
web-llm copied to clipboard
[Bug] simple-chat example works only with llama2 models, crashes with redpajama and vicuna
Install and run the current simple-chat example, I used both node v18 and v20. While it works with the llama-2 models, using the Redpajama or Vicuna models will generate this error in the dev tools console:
FATAL] /Users/ruihang-macstudio/Workspace/tvm/src/runtime/relax_vm/builtin.cc:177: Check failed: (ptr->dl_tensor.ndim == ndim) is false: ValueError: ErrorContext(fn=prefill, loc=param[3], param=params, annotation=R.Tuple(R.Tensor((50432, 320), dtype="uint32"), R.Tensor((50432, 80), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,),
When it starts the server for the first time - there were two error messages:
[Error: ENOENT: no such file or directory, open '/home/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'] {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/home/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'
Installing url...
⠋ Building _empty.js...
[Error: ENOENT: no such file or directory, open '/home/autoqa/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'] {
errno: -2,
code: 'ENOENT',
syscall: 'open',
This is so strange. It works with latest Edge (Bing) but not latest chrome 🙄
yet the chat at https://webllm.mlc.ai/ works, when accessed on the same Chrome browser that crashed with simple-chat 🤔
I met the same situation.