执行python -m web_demo.web_ability_demo demo_VITA_ckpt/命令出现如下这个报错

[youxingtian]

Uninitialized parameters: ['model.audio_encoder.encoder.global_cmvn.istd', 'model.audio_encoder.encoder.global_cmvn.mean'] INFO 01-14 15:36:57 model_runner.py:1052] Loading model weights took 15.5767 GB WARNING 01-14 15:36:57 model_runner.py:1223] Computed max_num_seqs (min(256, 32768 // 182272)) to be less than 1. Setting it to the minimum value of 1. INFO 01-14 15:36:58 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20250114-153658.pkl... INFO 01-14 15:36:59 model_runner_base.py:149] Completed writing input of failed execution to /tmp/err_execute_model_input_20250114-153658.pkl. [rank0]: Traceback (most recent call last): [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/worker/model_runner_base.py", line 116, in _wrapper [rank0]: return func(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1622, in execute_model [rank0]: hidden_or_intermediate_states = model_executable( [rank0]: File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl [rank0]: return self._call_impl(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl [rank0]: return forward_call(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 1124, in forward [rank0]: image_input = self._parse_and_validate_image_input(**kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 983, in _parse_and_validate_image_input [rank0]: data=self._validate_pixel_values(pixel_values), [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 965, in _validate_pixel_values [rank0]: _validate_shape(d) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 959, in _validate_shape [rank0]: raise ValueError( [rank0]: ValueError: The expected shape of pixel values per image per batch per patch is (3, 448, 448). You supplied (256, 3, 448, 448).

[rank0]: The above exception was the direct cause of the following exception:

[rank0]: Traceback (most recent call last): [rank0]: File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main [rank0]: return _run_code(code, main_globals, None, [rank0]: File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code [rank0]: exec(code, run_globals) [rank0]: File "/data/VITA/web_demo/web_ability_demo.py", line 519, in [rank0]: main(args.model_path) [rank0]: File "/data/VITA/web_demo/web_ability_demo.py", line 498, in main [rank0]: llm = LLM( [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/llm.py", line 214, in init [rank0]: self.llm_engine = LLMEngine.from_engine_args( [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 564, in from_engine_args [rank0]: engine = cls( [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 339, in init [rank0]: self._initialize_kv_caches() [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 474, in _initialize_kv_caches [rank0]: self.model_executor.determine_num_available_blocks()) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/executor/gpu_executor.py", line 114, in determine_num_available_blocks [rank0]: return self.driver_worker.determine_num_available_blocks() [rank0]: File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 223, in determine_num_available_blocks [rank0]: self.model_runner.profile_run() [rank0]: File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1263, in profile_run [rank0]: self.execute_model(model_input, kv_caches, intermediate_tensors) [rank0]: File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context [rank0]: return func(*args, **kwargs) [rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/worker/model_runner_base.py", line 152, in _wrapper [rank0]: raise type(err)( [rank0]: ValueError: Error in model execution (input dumped to /tmp/err_execute_model_input_20250114-153658.pkl): The expected shape of pixel values per image per batch per patch is (3, 448, 448). You supplied (256, 3, 448, 448). I0114 15:36:59.673532 21097 ProcessGroupNCCL.cpp:1126] [PG 0 Rank 0] ProcessGroupNCCL destructor entered. I0114 15:36:59.673600 21097 ProcessGroupNCCL.cpp:1111] [PG 0 Rank 0] Launching ProcessGroupNCCL abort asynchrounously. I0114 15:36:59.673952 21097 ProcessGroupNCCL.cpp:1016] [PG 0 Rank 0] future is successfully executed for: ProcessGroup abort I0114 15:36:59.673964 21097 ProcessGroupNCCL.cpp:1117] [PG 0 Rank 0] ProcessGroupNCCL aborts successfully. I0114 15:36:59.673980 21097 ProcessGroupNCCL.cpp:1149] [PG 0 Rank 0] ProcessGroupNCCL watchdog thread joined. I0114 15:36:59.674026 21097 ProcessGroupNCCL.cpp:1153] [PG 0 Rank 0] ProcessGroupNCCL heart beat monitor thread joined.

[lxysl]

The root cause of your issue is:

[rank0]: File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 959, in _validate_shape
[rank0]: raise ValueError(
[rank0]: ValueError: The expected shape of pixel values per image per batch per patch is (3, 448, 448). You supplied (256, 3, 448, 448).

It appears that you did not follow the instructions in the README file correctly. I guess you might not have executed the following instructions:

# Backup a new weight file
cp -rL  VITA_ckpt/ demo_VITA_ckpt/

mv demo_VITA_ckpt/config.json demo_VITA_ckpt/origin_config.json

cd ./web_demo/vllm_tools
cp -rf qwen2p5_model_weight_file/*  ../../demo_VITA_ckpt/
cp -rf vllm_file/*  your_anaconda/envs/vita_demo/lib/python3.10/site-packages/vllm/model_executor/models/

Please check and provide feedback on whether this is the issue.

[youxingtian]

感谢，目前这个报错已经解决，我用vllm0.6.2版本的，这个项目使用的是vllm0.5.5版本