TypeError: MiniCPMV.forward() missing 1 required positional argument: 'data'

[rueing]

Checklist

• [ ] 1. I have searched related issues but cannot get the expected help.
• [ ] 2. The bug has not been fixed in the latest version.
• [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

Python 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0] on linux Type "help", "copyright", "credits" or "license" for more information.

from lmdeploy import pipeline from lmdeploy.vl import load_image

pipe = pipeline('/mnt/music-live-common/llmModelZone/MiniCPM/MiniCPM-Llama3-V-2_5') Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:08<00:00, 1.24s/it] image = load_image('https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg') image <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=278x182 at 0x7F47832E0450> response = pipe(('describe this image', image)) 2024-07-31 21:58:44,625 - lmdeploy - ERROR - Engine loop failed with error: MiniCPMV.forward() missing 1 required positional argument: 'data' Traceback (most recent call last): File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/request.py", line 17, in _raise_exception_on_finish task.result() File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 857, in async_loop await self._async_loop() File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 847, in _async_loop await __step(True) File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 833, in __step raise e File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 825, in __step raise out File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 774, in _async_loop_background await self._async_step_background( File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 683, in _async_step_background output = await self._async_model_forward(inputs, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/utils.py", line 253, in __tmp return (await func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 590, in _async_model_forward return await __forward(inputs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 568, in __forward return await self.model_agent.async_forward( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 756, in async_forward output = self._forward_impl(inputs, ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 723, in _forward_impl output = model_forward( ^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/engine/model_agent.py", line 497, in model_forward output = patched_model.patched_forward( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/gitProject/lmdeploy/lmdeploy/pytorch/models/patch.py", line 210, in call output = self._model(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/condaEnv/lmdeploy/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/music-live-common/heying/condaEnv/lmdeploy/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: MiniCPMV.forward() missing 1 required positional argument: 'data'

Reproduction

不知道什么原因会报这个错？

Environment

不知道什么原因会报这个错？

Error traceback

No response

[irexyc]

lmdeploy 有两个后端，turbomind 和 pytorch，这个模型只在turbomind后端支持。

从你的日志中看是应该是没安装好。lmdeploy 不支持 pip install . 这种安装方式无法安装turbomind后端，请使用 pip install lmdeploy

可以打开日志看更多的信息 pipe = pipeline('/mnt/music-live-common/llmModelZone/MiniCPM/MiniCPM-Llama3-V-2_5', log_level='INFO')

可以lmdeploy check_env 打印一些pyhon 环境信息。

[rueing]

uninstall之后，重新pip install lmdeploy。依然还是这个报错。TypeError: MiniCPMV.forward() missing 1 required positional argument: 'data' 不知道是什么情况了...

[irexyc]

pipe = pipeline('/mnt/music-live-common/llmModelZone/MiniCPM/MiniCPM-Llama3-V-2_5', log_level='INFO')

日志是怎样的？

[github-actions[bot]]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

[jaytsien]

@irexyc 你好，我再minicpm-v2.6遇到相同的问题，想问下这个是必须得等发版才能用么，有什么方式能够提前使用新特性

[irexyc]

@jaytsien

https://github.com/zhyncs/lmdeploy-build/releases/tag/ada7285 可以用nightly build，不过正常情况不应该出现使用pytorch backend加载的情况，可以把日志打开看下。

[github-actions[bot]]

This issue is marked as stale because it has been marked as invalid or awaiting response for 7 days without any further response. It will be closed in 5 days if the stale label is not removed or if there is no further response.

[github-actions[bot]]

This issue is closed because it has been stale for 5 days. Please open a new issue if you have similar issues or you have any new updates now.