JerryYao80

Error : model.embed_tokens.weight

2

Download the latest source code, runing the command : "python -m fastchat.serve.cli --model-path "lmsys/vicuna-13b-delta-v1.1" --load-8bit" error occurs: init_kwargs {'torch_dtype': torch.float16} 0it [00:00, ?it/s] ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮...

Output of quantized Vicuna is so inappropriate that I can't use it

4

# Prerequisites Please answer the following questions for yourself before submitting an issue. - [√] I am running the latest code. Development is very rapid so there are no tagged...