FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Add `seed` parameter support for openai api protocol. I tested vllm_worker: ```bash python3 -m fastchat.serve.controller CUDA_VISIBLE_DEVICES=0 python3 -m fastchat.serve.vllm_worker --model-path facebook/opt-125m --model-names opt-125m --controller http://localhost:21001 --port 31001 --worker-address http://localhost:31001 python3...
`from fastchat.serve.api_provider import init_palm_chat` The code above has a problem "**Cannot find reference 'init_palm_chat' in 'api_provider.py'** " I can't find anything about init_palm_chat from the file "api_provider ". How to...
I think it's necessary to provide a security vulnerability report for the project [https://github.com/lm-sys/FastChat/blob/main/fastchat/model/model_adapter.py#L108](https://github.com/lm-sys/FastChat/blob/main/fastchat/model/model_adapter.py#L108) ` tokenizer = AutoTokenizer.from_pretrained( model_path, use_fast=self.use_fast_tokenizer, revision=revision, trust_remote_code=True, ) ` There is a security risk in...
## Why are these changes needed? Critique Labs AI has developed an agentic search engine that we'd like to add to the search arena. This PR is the implementation of...
While this works on VLLM 0.7.x, the latest one (0.8.2), which supports mistral-small and gemma, does not seem have the "engine" attribute: ```python 2025-03-24 22:04:27 | ERROR | stderr |...
could you explain the purpose make the n=1 fixed I want to call the vllm worker directly, but I can not get multipe choice. I checked the code, find this...
This makes vllm_worker work out of the box with vllm v0.8.3, which opens FastChat to work with all the recent models! @merrymercy it's just a environment variable ## Why are...
## Summary Adds video arena leaderboard from https://videoarena.tv/leaderboard to FastChat leaderboard. ## Checks - [x] I've run `format.sh` to lint the changes in this PR. - [x] I've included any...
Since december, DashInfer has GPU support, and the dashinfer_worker doesn't take that into account. @yejunjin what do you think about a refresh on this worker? :-)
There is an import from lightllm which changed location.