InternLM
[Bug] 使用v0.10.1版本部署internvl2.5-38b-mpo-awq报错，如何解决？
Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
Describe the bug
/home/li_mingze/.local/lib/python3.12/site-packages/torch/cuda/init.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]
Traceback (most recent call last):
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2302, in getattr
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2332, in _get_module
raise e
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/utils/import_utils.py", line 2330, in _get_module
return importlib.import_module("." + module_name, self.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/data3/mingze/miniforge3/envs/eric_lmdeploy/bin/lmdeploy", line 7, in
Reproduction
CUDA_VISIBLE_DEVICES=0,1 lmdeploy serve api_server /data3/mingze/models/OpenGVLab/InternVL2_5-38B-MPO-AWQ --server-port 2556 --cache-max-entry-count 0.5 --tp 2 --backend pytorch > /data3/mi ngze/big_test/38_lmdeploy.log 2>&1 &
Environment
Package Version
--------------------------------- -------------
accelerate 1.10.1
addict 2.4.0
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiosignal 1.4.0
annotated-types 0.7.0
anyio 4.10.0
astor 0.8.1
attrs 25.3.0
blake3 1.0.5
cachetools 6.2.0
cbor2 5.7.0
certifi 2025.8.3
charset-normalizer 3.4.3
click 8.2.1
cloudpickle 3.1.1
compressed-tensors 0.11.0
cupy-cuda12x 13.6.0
depyf 0.19.0
dill 0.4.0
diskcache 5.6.3
distro 1.9.0
dnspython 2.8.0
einops 0.8.1
email-validator 2.3.0
fastapi 0.116.2
fastapi-cli 0.0.11
fastapi-cloud-cli 0.1.5
fastrlock 0.8.3
filelock 3.19.1
fire 0.7.1
frozenlist 1.7.0
fsspec 2025.9.0
genson 1.3.0
gguf 0.17.1
h11 0.16.0
hf-xet 1.1.10
httpcore 1.0.9
httptools 0.6.4
httpx 0.28.1
huggingface-hub 0.35.0
idna 3.10
interegular 0.3.3
Jinja2 3.1.6
jiter 0.11.0
jsonpath-ng 1.7.0
jsonschema 4.25.1
jsonschema-specifications 2025.9.1
lark 1.2.2
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.11.3
lmdeploy 0.10.1
markdown-it-py 4.0.0
MarkupSafe 3.0.2
mdurl 0.1.2
mistral_common 1.8.5
mmengine-lite 0.10.7
mpmath 1.3.0
msgpack 1.1.1
msgspec 0.19.0
multidict 6.6.4
networkx 3.5
ninja 1.13.0
numba 0.61.2
numpy 2.2.6
nvidia-cublas-cu12 12.8.4.1
nvidia-cuda-cupti-cu12 12.8.90
nvidia-cuda-nvrtc-cu12 12.8.93
nvidia-cuda-runtime-cu12 12.8.90
nvidia-cudnn-cu12 9.10.2.21
nvidia-cufft-cu12 11.3.3.83
nvidia-cufile-cu12 1.13.1.3
nvidia-curand-cu12 10.3.9.90
nvidia-cusolver-cu12 11.7.3.90
nvidia-cusparse-cu12 12.5.8.93
nvidia-cusparselt-cu12 0.7.1
nvidia-ml-py 13.580.82
nvidia-nccl-cu12 2.27.3
nvidia-nvjitlink-cu12 12.8.93
nvidia-nvtx-cu12 12.8.90
nvitop 1.5.3
openai 1.107.3
openai-harmony 0.0.4
opencv-python-headless 4.12.0.88
outlines 1.2.5
outlines_core 0.2.11
packaging 25.0
partial-json-parser 0.2.1.1.post6
peft 0.14.0
pillow 11.3.0
pip 25.2
platformdirs 4.4.0
ply 3.11
prometheus_client 0.22.1
prometheus-fastapi-instrumentator 7.1.0
propcache 0.3.2
protobuf 6.32.1
psutil 7.0.0
py-cpuinfo 9.0.0
pybase64 1.4.2
pycountry 24.6.1
pydantic 2.11.9
pydantic_core 2.33.2
pydantic-extra-types 2.10.5
Pygments 2.19.2
pynvml 13.0.1
python-dotenv 1.1.1
python-json-logger 3.3.0
python-multipart 0.0.20
PyYAML 6.0.2
pyzmq 27.1.0
ray 2.49.1
referencing 0.36.2
regex 2025.9.1
requests 2.32.5
rich 14.1.0
rich-toolkit 0.15.1
rignore 0.6.4
rpds-py 0.27.1
safetensors 0.6.2
scipy 1.16.2
sentencepiece 0.2.1
sentry-sdk 2.38.0
setproctitle 1.3.7
setuptools 79.0.1
shellingham 1.5.4
shortuuid 1.0.13
sniffio 1.3.1
soundfile 0.13.1
soxr 1.0.0
starlette 0.48.0
sympy 1.14.0
termcolor 3.1.0
tiktoken 0.11.0
timm 1.0.19
tokenizers 0.22.0
torch 2.8.0
torchaudio 2.8.0
torchvision 0.23.0
tqdm 4.67.1
transformers 4.56.1
triton 3.4.0
typer 0.17.4
typing_extensions 4.15.0
typing-inspection 0.4.1
urllib3 2.5.0
uvicorn 0.35.0
uvloop 0.21.0
vllm 0.10.2
watchfiles 1.1.0
websockets 15.0.1
wheel 0.45.1
xformers 0.0.32.post1
xgrammar 0.1.23
yapf 0.43.0
yarl 1.20.1
Error traceback
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/models/auto/processing_auto.py", line 28, in
from ...processing_utils import ProcessorMixin
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/processing_utils.py", line 34, in
from .audio_utils import load_audio
File "/home/li_mingze/.local/lib/python3.12/site-packages/transformers/audio_utils.py", line 42, in
import soundfile as sf
File "/home/li_mingze/.local/lib/python3.12/site-packages/soundfile.py", line 17, in
from _soundfile import ffi as _ffi
File "/home/li_mingze/.local/lib/python3.12/site-packages/_soundfile.py", line 2, in
import _cffi_backend
ModuleNotFoundError: No module named '_cffi_backend'
从这里看似乎是你的环境里面的 soundfile 没有安装好，导致导入 AutoProcessor 失败
可以验证一下环境里面的包是否安装好了
python -c "import soundfile as sf"
