FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Abu Dhabi's Technology Innovation Institute (TII) just released new 7B and 40B LLMs. The Falcon-40B model is now at the top of the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), beating *llama-30b-supercot* and *llama-65b*...
GPUs: 2 x RTX 4090 24G run below command: python3 -m fastchat.serve.cli --model-path ~/vicuna-13b-1_1-hf/ --num-gpus 2 All it's ok before i input prompt, I tried to input "hello", get below...
Vicuna was build and worked as main.cpp by mself. But after this, I installed fastchat and it can't use the vicuna.cpp model file. Is it possible to fix this? ```...
Conversation template has existing values for messages. This will cause token count to increase and could lead to confusing responses from the model. Resolved it by `conv.messages=[]`.
## Why are these changes needed? [Feature Enhancement] Improve SSE User Experience This PR aims to enhance the user experience of SSE (Server-Sent Events). The following changes have been made:...
Add LoRA adapter, so that a finetuned adapter doesn't need to be merged to model every time, but instead be loaded directly. ## Why are these changes needed? Loading Adapters...
## Why are these changes needed? We are going to use [xformer](https://github.com/facebookresearch/xformers) instead of flash attention. Xformer is better because: - It supports more GPU architectures than flash attention, including...
Hi Vicuna authors, I appreciate your excellent work in making it public. I am curious if you could provide a demo code for using vicuna to do the inference in...
at first we edit the dummy.json file, changed the "my name is Vicuna" as "my name is XXXXX", and keep all the other conversations (total 910) , then trained it,...
