web-llm icon indicating copy to clipboard operation
web-llm copied to clipboard

[Bug] simple-chat example works only with llama2 models, crashes with redpajama and vicuna

Open Sing-Li opened this issue 2 years ago • 3 comments

Install and run the current simple-chat example, I used both node v18 and v20. While it works with the llama-2 models, using the Redpajama or Vicuna models will generate this error in the dev tools console:

FATAL] /Users/ruihang-macstudio/Workspace/tvm/src/runtime/relax_vm/builtin.cc:177: Check failed: (ptr->dl_tensor.ndim == ndim) is false: ValueError: ErrorContext(fn=prefill, loc=param[3], param=params, annotation=R.Tuple(R.Tensor((50432, 320), dtype="uint32"), R.Tensor((50432, 80), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), dtype="float32"), R.Tensor((320, 2560), dtype="uint32"), R.Tensor((80, 2560), dtype="uint32"), R.Tensor((2560,), 

When it starts the server for the first time - there were two error messages:

[Error: ENOENT: no such file or directory, open '/home/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/home/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'
Installing url...
⠋ Building _empty.js...
[Error: ENOENT: no such file or directory, open '/home/autoqa/dev/web-llm/examples/simple-chat/node_modules/@mlc-ai/web-llm/node_modules/@mlc-ai/web-tokenizers/lib/index.js'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',

Sing-Li avatar Aug 12 '23 05:08 Sing-Li

This is so strange. It works with latest Edge (Bing) but not latest chrome 🙄

Sing-Li avatar Aug 12 '23 05:08 Sing-Li

yet the chat at https://webllm.mlc.ai/ works, when accessed on the same Chrome browser that crashed with simple-chat 🤔

Sing-Li avatar Aug 12 '23 14:08 Sing-Li

I met the same situation.

ToSeven avatar Oct 15 '23 13:10 ToSeven