FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
where is the training data for the model that it was trained on? is this in a git repo or on huggingface somepalce?
Which version of gpt-4 is used to generate the mt-bench scores on lmsys leaderboard https://chat.lmsys.org/?leaderboard ? Is it gpt-4-0613 or gpt-4-0314?
In a multi-threaded situation, if the GPU server resources are insufficient, will cache kv preemption occur? For example, there are two conversations at the same time, both of which are...
``` python3 -m fastchat.serve.cli --model-path /home/models/Qwen1.5-32B-Chat --device npu --gpus 0,1,2,3 (fast_chat) [root@localhost ~]# python3 -m fastchat.serve.cli --model-path /home/models/Qwen1.5-32B-Chat --device npu --gpus 0,1,2,3 /root/miniconda3/envs/fast_chat/lib/python3.8/site-packages/torch_npu/dynamo/__init__.py:18: UserWarning: Register eager implementation for the 'npu'...
for example : `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 --num-gpus 2` maybe this following command need to be supported? `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 --device npu --num-npus 2`  
Could you add support NousResearch/Hermes-2-Pro-Mistral-7B model
命令： python -m fastchat.serve.cli --model-path ~/data/model/chatglm3-6b --gpus 2 报错信息： Traceback (most recent call last): File "/root/miniconda3/envs/ragllm/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/root/miniconda3/envs/ragllm/lib/python3.10/runpy.py", line 86, in _run_code...
Hi, I'm a blind user, and I can't access leaderboard.lmsys.org with a screen reader. I tried accessing the website with many screen readers including Jaws, NVDA, Narrator on Windows, and...
## Why are these changes needed? I need save the battle logs in the `conv.json`, and then I found that garbled Chinese in the file. The reason is `json.dumps` the...
## Why are these changes needed? [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM) can greatly improve the inference speed of LLM. It would be helpful to support tensorRT-LLM in Fastchat. This commit simply implements how to...
