pppppM

[Refactor & Feature] Refactor `xtuner chat` to support `lmdeploy` &`vLLM`

1

# Motivation - 可以对接推理引擎加速 `xtuner chat` - 支持 xtuner 训练得到的模型直接部署 - 方便 xtuner 开发 gradio 应用 - 保证训练部署时对话模板一致 - 简化部署流程 # Usage 1. xtuner chat 启动命令 ``` # HF python...

1. [x] Verify on the local Linux machine 2. [ ] Verify on Windows WSL 3. [x] Push to Docker Hub (docker pull pppppm/xtuner:0.1.9dev) 4. [ ] Verify on Aliyun

- [x] get_started - [x] training - [x] accelerate - [ ] InternEVO

Wechat Group

1

Thanks for your attention ! If you want to distill model in OpenMMLab related repos, could join the wechat group in README.md

In XTuner Lite, some modifications were made to HF Llava to support the LLM part which is remote code. As a result, the saved HF Llava model needs to be...