suc16

I have the same issue with fastchat 0.2.1. I have tried to update huggingface transformers and restart workers, but still not work. vicuna v0 and vicuna v1.1 both have the...

> New models and v.0.1.10 works for me so - fschat-v0.1.10 + vicuna-7b-v0 work - fschat-v0.1.10 + vicuna-7b-v1.1 work - fschat-v0.2.1 + vicuna-7b-v0 not work - fschat-v0.2.1 + vicuna-7b-v1.1 not...

Thanks. My environment python3.9 transformers 4.28.1 fschat 0.2.2 After applying delta with latest fastchat, I still get the blank EOS/BOS in special_tokens_map.json python3 -m fastchat.model.apply_delta --base /data/models/llama-7b-hf --target /data/models/vicuna-7b --delta...

> > > 我这个训练速度正常吗，18w的数据，batch_size=64, 2轮，居然要32个小时，V100 80GGPU，合着一秒钟才5条文本？ A100才有80g吧。。。这个速度似乎是正常的。

> > > > 写错了，是A100，这可真慢啊，我发现增加batchsize对于加速一点用没有 你的代码里batchsize大了，max_steps是会按照比例下调是吧。

> 资源没利用起来，很多你定义的参数其实是无效的，这里推荐一篇blog：[ Efficient Training on a Single GPU](https://huggingface.co/docs/transformers/perf_train_gpu_one#efficient-training-on-a-single-gpu) > > 另外可以改进的包括但不限于： > > ``` > bf16 = True > tf32 = True > optim = “adamw_torch_fused” # 或者安装apex后 "adamw_apex_fused" >...

参考官方库格式拼接history prompt = "" for i, (old_query, response) in enumerate(history): prompt += "[Round {}]\n问：{}\n答：{}\n".format(i, old_query, response) prompt += "[Round {}]\n问：{}\n答：".format(len(history), query)

> ```python > from fastapi import FastAPI, Request > from transformers import AutoTokenizer, AutoModel > import uvicorn > import json > import datetime > import torch > from peft import...

> @Ling-yunchi > > 大佬太强了！要是能提gradio更好啦 类似大佬这个前后端分离的，可以去看看fastchat

> @suc16 fastchat？？？ 应该叫参考一下fastchat fastchat的这个server，应该是前后端分离的比较好的 https://github.com/lm-sys/FastChat/blob/main/fastchat/serve/gradio_web_server.py