inference
inference copied to clipboard
[feature request] MiniCPM-2B-sft-bf16 support vllm inference
Feature request / 功能建议
MiniCPM-2B-sft-bf16 not support vllm in xinference. It's actually support vllm officially: https://github.com/OpenBMB/MiniCPM/blob/main/inference/inference_vllm.py and https://docs.vllm.ai/en/latest/models/supported_models.html
Motivation / 动机
please add vllm support for MiniCPM-2B-sft-bf16, openbmb/MiniCPM-2B-dpo-bf16, etc.
Your contribution / 您的贡献
NA
OK, we will support it ASAP.
This issue is stale because it has been open for 7 days with no activity.
This issue was closed because it has been inactive for 5 days since being marked as stale.