FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
When using the SHORTEST_QUEUE method for load balancing, an issue arises because the queue_length isn't updated when a generation task ends. This leads to inaccurate load balancing across worker processes....
## Why are these changes needed? Fix the bugs when `sampling_weights` is empty; Fix the bugs when `random_questions` is empty ## Related issue number (if applicable) ## Checks - [x]...
If it is not appropriate to list it as an Issue, please point it out. To Arena, I would like you to add the following model. Model 9B gemma-2, but...
## using `dumps_kwargs` keyword arguments are no longer supported. while deploying and testing qwen1.5-7b-chat through fastchat v0.2.36, i got an exception : ``` 2024-08-16 03:11:13 | INFO | stdout |...
## Why are these changes needed? In the `fastchat/train/train.py` file, I found a repeated assignment of the tokenizer in the `LazySupervisedDataset` class. ```python class LazySupervisedDataset(Dataset): """Dataset for supervised fine-tuning.""" def...
## Why are these changes needed? This pull request adds support for loading models in 4-bit quantized versions. This enhancement addresses the need for more efficient model loading and storage,...
代码： #!/bin/bash # 启动控制器 python -m fastchat.serve.controller --host 0.0.0.0 --port 2002 & # 为模型设置环境变量并启动 export CUDA_VISIBLE_DEVICES=1 python -m fastchat.serve.model_worker --model-path ./bge-large-zh-v1.5 --model-names gpt-4 --controller-address http://0.0.0.0:2002 & # 启动openai服务器 python -m...
## Why are these changes needed? This PR fixes critical issues in image format conversion: - Undefined `image` variable - Ensured proper initialization for URL/local/bytes formats - Method argument mismatch...
