FastChat
FastChat copied to clipboard
lm-sys
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
## Why are these changes needed? Support llama3 fine-tune, which is the extension of https://github.com/lm-sys/FastChat/pull/3259. Also, the length-1 tokenization mismatch is fixed. ## Related issue number (if applicable) ## Checks...
Hi， my env as belows : docker image : docker run --gpus all -it --net=host --ipc=host --ulimit memlock=-1 -v /home/ubuntu/test:/home/finetune -v /ssd/gyou:/models --name=vicuna nvcr.io/nvidia/pytorch:23.07-py3 run command : root@g0300:/home/finetune/FastChat# cat ./scripts/train_vicuna_13b.sh...
## Why are these changes needed? Filters out PII from chat logs using Azures PII detection ## Related issue number (if applicable) ## Checks - [ X] I've run `format.sh`...
Now that many newer Huggingface models come with a chat template in their tokenizer, FastChat should use it as the primary way to build conversations, falling back to `conversation.py` when...
Use FastChat to start the deepseek-coder-33b-instruct model, send a stream request and got an error response. If set stream=False, you can print a good response If change to other models,...
## Why are these changes needed? ## Related issue number (if applicable) ## Checks - [ ] I've run `format.sh` to lint the changes in this PR. - [ ]...
I tried to test the MT-bench score for the Meta-Llama3-8B-Instruct model, and it keeps popping maximum context length errors This model's maximum context length is 8192 tokens. However, your messages...
## Why are these changes needed? Solve the compatibility problem of SeparatorStyle.CHATML type messes field /usr/local/lib/python3.10/dist-packages/fastchat/conversation.py", line 197, in get_prompt ERROR | stderr | ret += role + ":" +...
