FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
I have a peft adapter model for a finetuned Falcon7b model, When using gen_mode_answer.py, i get this error: TypeError: PeftModelForCausalLM.generate() takes 1 positional argument but 2 were given python gen_model_answer.py...
fschat==0.2.36 ``` WARNING 02-29 16:06:12 config.py:140] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models. INFO 02-29 16:06:12 llm_engine.py:72] Initializing an LLM engine with...
when I run ' python3 -m fastchat.serve.openai_api_server --host localhost --port 8000' then output error : Traceback (most recent call last): File "/work/miniconda3/envs/zq_vllm/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None,...
## Why are these changes needed? When using langchaingo w/ fastchat, it generates requests with formats like `"messages":["text":"given text", "type": "text"]` as default, which will be recognized and processed as...
Is FastChat not support M3E-BASE？ it can run bge-m3 but can't run M3E-BASE
I used the local model about 40GB, then increased it to 50GB as the coversation increased, but couldn't free the GPU when exited the conversations
Hello,I used Qlora to train,but I get an error: NotImplementedError: Cannot copy out of meta tensor; no data! requriements.txt: ``` peft @ file:///root/peft torch==1.13.1+cu116 torchaudio==0.13.1+cu116 torchvision==0.14.1+cu116 transformers==4.28.1 deepspeed==0.9.4 flash-attn==0.2.0 ```...
## Why are these changes needed? Some models need to specifically learn to generate the stop tokens. Otherwise these trained models will not stop when serving. This is a model...
Hey. I haven't found similar examples for using FastChat for multi-node training. My script is as follows: I have found from this website https://pytorch.org/docs/stable/elastic/run.html to use the "rdzv-id, rdzv-backend and...
## Why are these changes needed? With this feature, we can use the guidance to controll fastchat's llm output. See this [issue2481](https://github.com/lm-sys/FastChat/issues/2481). ## Related issue number (if applicable) [issue2481](https://github.com/lm-sys/FastChat/issues/2481). ##...
