FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
**current fastchat version:** 0.2.36 I served the embedding model in openai style with the code below and I want to connet with LangChain, but errors occured! how to deal with...
Currently the `--max-model-len` can be passed to vllm through `kwarg`, but could it be added to the default param like `gpu-utilization-limit`? It is often needed when using models that can...
Hi, Thank you for releasing the Arena. Which model is `gpt2-chatbot`? Thanks!
Hi! I'm using the default configuration in llm_judge repo. But when I use openai apis from different mirror, I got significantly different result. I use llama3-8b, and the score from...
This PR includes basic code for analyzing the time it takes to vote using collected data. ## Checks - [x] I've run `format.sh` to lint the changes in this PR....
## Why are these changes needed? Converts the current imagebox + textbox setup to just use the multimodal textbox from gradio. ## Related issue number (if applicable) ## Checks -...
When just change ```--port``` , it has no error but it does not work well when run sevural models in one machine. And this does not change the behavior when...
https://github.com/THUDM/ChatGLM2-6B/issues/308 Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 261, in wrap await func() File "/usr/local/lib/python3.10/dist-packages/starlette/responses.py", line 250, in stream_response async for chunk in self.body_iterator: File "/usr/local/lib/python3.10/dist-packages/fastchat/serve/openai_api_server.py", line 506, in...
## Why are these changes needed? ## Related issue number (if applicable) ## Checks - [ ] I've run `format.sh` to lint the changes in this PR. - [ ]...
