FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Published 20 hours ago • verified publisher icon lm-sys

execute fastchat.serve.model_ Worker Error

Open ch930410 opened this issue 1 year ago • 4 comments

command: python -m fastchat.serve.model_worker --model-path /model_worker/lmsys/vicuna-13b-delta-v1.1

Error Info: image

image

ch930410 avatar Apr 19 '23 03:04 ch930410

Have the same issue. For now, FastChat is not available AT ALL due this this error on the worker side. Even more, FastChat response displayers weird words which are totally non-sense and not related to the questions.

yunghoy avatar Apr 20 '23 06:04 yunghoy

I have re read the document content and it seems that my model needs to be converted, and there is also a problem with my command. I am currently working on it and I am not sure if it is feasible.

ch930410 avatar Apr 20 '23 06:04 ch930410

Seems like a windows issue. i had the same error as well. But was able to run the converted model in Linux containers

Chesterguan avatar May 02 '23 15:05 Chesterguan

You are using delta weights, You need to convert it to the HF format with original LLama Weights

hardikvirani6 avatar May 30 '23 11:05 hardikvirani6

@ch930410 this looks like a non-issue - it was a problem of the model version, not of fastChat. And now even less with the latest vicunas based on llama2. Let's close this one?

surak avatar Oct 21 '23 15:10 surak