FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
from OpenAI 1.26.0, Usage stats now available when using streaming with the Chat Completions API or Completions API https://community.openai.com/t/usage-stats-now-available-when-using-streaming-with-the-chat-completions-api-or-completions-api/738156 FastChat should also support this stat.
## Why are these changes needed? When adding the first user flag (with a message of "None"), and then check the conversation prompt like this: ```python conv_template.append_message(conv_template.roles[0], None) conv_template.get_prompt() ```...
Hi, I am trying to add embedding function. To achieve it, I have read docs/langchain_integration.md, docs/openai_api.md, tests/test_openai_langchain.py. I am not familiar with web deployment so I am a little lost....
Nvidia has their own solution to deploy large language models. Would it make sense to have an adapter? https://developer.nvidia.com/blog/deploy-multilingual-llms-with-nvidia-nim
When using fastchat to deploy the openapi_api interface, calling "/v1/models" returns normal results, but calling "/v1/chat/completions" reports the following errors： INFO: 192.168.0.2:53398 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request 2024-07-02...
flag preset images more easily. note: at some point we probably want to move these image features to the image class and create a todict() class for that
Does the online ChatArena have any filtering mechanism to filter the following queries where the user will ask the model who they are? It may cause cheating voting.  
## 问题描述 使用第2第3块gpu启动时，卡着不动（而使用1 2、1 3的两两组合则没有问题） cuda版本：12.1.0 Driver版本: 535.54.03 torch: 2.1.2 fschat: 0.2.34 vllm: 0.2.6 ray: 2.8.1 ## 启动命令 ```shell CUDA_VISIBLE_DEVICES="2,3" python -m fastchat.serve.vllm_worker \ --model-names="qwen-72b-chat" \ --model-path="/Models/Qwen-72B-Chat" \ --controller-address=${CONTROLLER_ADDRESS} \...
Must provide either the `api_version` argument or the `OPENAI_API_VERSION` environment variable
``` 今天（2023年6月29日）星期几为星期四。 USER: 而且4月6号是 星期四 # --> 是{ }星, here is not space, after pressing to special characters use delete, throws an error │ py:316 in _EncodeAsPieces │ │ │...
