PaddlePaddle
VLLM depolyment of paddleVL shows ERROR 11-28 13:11:39 [dump_input.py:79] Dumping scheduler output for model execution
🔎 Search before asking
- [x] I have searched the PaddleOCR Docs and found no similar bug report.
- [x] I have searched the PaddleOCR Issues and found no similar bug report.
- [x] I have searched the PaddleOCR Discussions and found no similar bug report.
🐛 Bug (问题描述)
model initialization works as expected but model fail to chat with messages when running demo according to https://docs.vllm.ai/projects/recipes/en/latest/PaddlePaddle/PaddleOCR-VL.html
EngineCore_DP0 pid=1025) INFO 11-28 12:33:20 [cuda.py:427] Using FLASH_ATTN backend. Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00, 1.07it/s] (EngineCore_DP0 pid=1025) INFO 11-28 12:33:21 [default_loader.py:314] Loading weights took 0.98 seconds (EngineCore_DP0 pid=1025) INFO 11-28 12:33:21 [gpu_model_runner.py:3334] Model loading took 1.8929 GiB memory and 1.201215 seconds (EngineCore_DP0 pid=1025) INFO 11-28 12:33:22 [gpu_model_runner.py:4084] Encoder cache will be initialized with a budget of 8192 tokens, and profiled with 2 image items of the maximum feature size. (EngineCore_DP0 pid=1025) INFO 11-28 12:33:27 [backends.py:631] Using cache directory: /home/jovyan/.cache/vllm/torch_compile_cache/21421504f4/rank_0_0/backbone for vLLM's torch.compile (EngineCore_DP0 pid=1025) INFO 11-28 12:33:27 [backends.py:647] Dynamo bytecode transform time: 3.70 s (EngineCore_DP0 pid=1025) INFO 11-28 12:33:28 [backends.py:210] Directly load the compiled graph(s) for dynamic shape from the cache, took 1.260 s (EngineCore_DP0 pid=1025) INFO 11-28 12:33:29 [monitor.py:34] torch.compile takes 4.96 s in total (EngineCore_DP0 pid=1025) INFO 11-28 12:33:29 [gpu_worker.py:359] Available KV cache memory: 59.82 GiB (EngineCore_DP0 pid=1025) INFO 11-28 12:33:29 [kv_cache_utils.py:1229] GPU KV cache size: 3,484,944 tokens (EngineCore_DP0 pid=1025) INFO 11-28 12:33:29 [kv_cache_utils.py:1234] Maximum concurrency for 131,072 tokens per request: 26.59x (EngineCore_DP0 pid=1025) 2025-11-28 12:33:29,894 - INFO - autotuner.py:256 - flashinfer.jit: [Autotuner]: Autotuning process starts ... (EngineCore_DP0 pid=1025) 2025-11-28 12:33:29,903 - INFO - autotuner.py:262 - flashinfer.jit: [Autotuner]: Autotuning process ends Capturing CUDA graphs (mixed prefill-decode, PIECEWISE): 100%|██████████| 51/51 [00:01<00:00, 33.73it/s] Capturing CUDA graphs (decode, FULL): 100%|██████████| 35/35 [00:00<00:00, 55.86it/s] (EngineCore_DP0 pid=1025) INFO 11-28 12:33:32 [gpu_model_runner.py:4240] Graph capturing finished in 3 secs, took 0.14 GiB (EngineCore_DP0 pid=1025) INFO 11-28 12:33:32 [core.py:250] init engine (profile, create kv cache, warmup model) took 10.61 seconds INFO 11-28 12:33:35 [llm.py:352] Supported tasks: ['generate'] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:72] Dumping input data for V1 LLM engine (v0.11.1) with config: model='/data/pe-task/ai_storage_gcp_dsocr_test/20251118/PaddleOCR-VL-0.9B', speculative_config=None, tokenizer='/data/pe-task/ai_storage_gcp_dsocr_test/20251118/PaddleOCR-VL-0.9B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=131072, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=/data/pe-task/ai_storage_gcp_dsocr_test/20251118/PaddleOCR-VL-0.9B, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'level': None, 'mode': <CompilationMode.VLLM_COMPILE: 3>, 'debug_dump_path': None, 'cache_dir': '/home/jovyan/.cache/vllm/torch_compile_cache/21421504f4', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': ['vllm::unified_attention', 'vllm::unified_attention_with_output', 'vllm::unified_mla_attention', 'vllm::unified_mla_attention_with_output', 'vllm::mamba_mixer2', 'vllm::mamba_mixer', 'vllm::short_conv', 'vllm::linear_attention', 'vllm::plamo2_mamba_mixer', 'vllm::gdn_attention_core', 'vllm::kda_attention', 'vllm::sparse_attn_indexer'], 'compile_mm_encoder': False, 'use_inductor': None, 'compile_sizes': [], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.FULL_AND_PIECEWISE: (2, 1)>, 'cudagraph_num_of_warmups': 1, 'cudagraph_capture_sizes': [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256, 272, 288, 304, 320, 336, 352, 368, 384, 400, 416, 432, 448, 464, 480, 496, 512], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {}, 'max_cudagraph_capture_size': 512, 'local_cache_dir': '/home/jovyan/.cache/vllm/torch_compile_cache/21421504f4/rank_0_0/backbone'}, (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] Dumping scheduler output for model execution: SchedulerOutput(scheduled_new_reqs=[NewRequestData(req_id=0,prompt_token_ids_len=2933,mm_features=[MultiModalFeatureSpec(data={'image_grid_thw': MultiModalFieldElem(modality='image', key='image_grid_thw', data=tensor([ 1, 108, 108]), field=MultiModalBatchedField()), 'pixel_values': MultiModalFieldElem(modality='image', key='pixel_values', data=tensor([[[[0.7500, 0.7500, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7422, 0.7344, 0.7422], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7344, 0.7344, 0.7422], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7344, 0.7344, 0.7422]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7656, 0.7656, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7422, 0.7500, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7500, 0.7500, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7500, 0.7500, 0.7578]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7578, 0.7578, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7422, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7422, 0.7422, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7422, 0.7422, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7422, ..., 0.7422, 0.7422, 0.7500]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[[0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7422, 0.7422, 0.7500, ..., 0.7500, 0.7578, 0.7578]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7500, ..., 0.7500, 0.7578, 0.7578]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7500, 0.7500], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7500, 0.7578, 0.7578]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[[0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7500, 0.7500, 0.7500, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.7578, 0.7578, 0.7578, ..., 0.7578, 0.7578, 0.7578]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[[0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4824]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4668, 0.4668, 0.4668, ..., 0.4668, 0.4668, 0.4668], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4668, 0.4668, 0.4668], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4668, 0.4668, 0.4668], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4746, 0.4746, 0.4746]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4512, 0.4512, 0.4512, ..., 0.4512, 0.4512, 0.4512], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4512, 0.4512, 0.4512], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4512, 0.4512, 0.4512], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4590, 0.4590, 0.4590]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[[0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4902], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4902], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4824, 0.4824, 0.4902]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4668, 0.4668, 0.4668, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4824]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4590, 0.4590, 0.4590, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4512, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4590, 0.4590, 0.4590, ..., 0.4590, 0.4434, 0.4512], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4590, 0.4590, 0.4590, ..., 0.4434, 0.4434, 0.4512], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4590, 0.4590, 0.4590, ..., 0.4434, 0.4434, 0.4512]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[[0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4824, 0.4824, 0.4902, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4902, 0.4902, 0.4902, ..., 0.4902, 0.4902, 0.4902], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4902, 0.4902, 0.4902, ..., 0.4902, 0.4902, 0.4902], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4902, 0.4902, 0.4902, ..., 0.4902, 0.4902, 0.4902]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4746, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4746, 0.4746, 0.4824, ..., 0.4746, 0.4746, 0.4746], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4824, 0.4824, 0.4824, ..., 0.4824, 0.4824, 0.4824]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [[0.4590, 0.4590, 0.4590, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4590, 0.4590, 0.4590, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4590, 0.4590, 0.4668, ..., 0.4590, 0.4590, 0.4590], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] ..., (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4668, ..., 0.4668, 0.4668, 0.4668], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4668, ..., 0.4668, 0.4668, 0.4668], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] [0.4512, 0.4512, 0.4668, ..., 0.4668, 0.4668, 0.4668]]]], (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [dump_input.py:79] dtype=torch.bfloat16), field=MultiModalBatchedField())}, modality='image', identifier='60a6bf2230042fb231557ecd823549a50ef5bacc69c8e00c18433abe998ac41d', mm_position=PlaceholderRange(offset=5, length=2916, is_embed=None))],sampling_params=SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.6, top_p=0.9, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=20480, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None),block_ids=([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184],),num_computed_tokens=0,lora_request=None,prompt_embeds_shape=None)], scheduled_cached_reqs=CachedRequestData(req_ids=[], resumed_req_ids=[], new_token_ids=[], all_token_ids={}, new_block_ids=[], num_computed_tokens=[], num_output_tokens=[]), num_scheduled_tokens={0: 2933}, total_num_scheduled_tokens=2933, scheduled_spec_decode_tokens={}, scheduled_encoder_inputs={0: [0]}, num_common_prefix_blocks=[184], finished_req_ids=[], free_encoder_mm_hashes=[], pending_structured_output_tokens=false, kv_connector_metadata=null, ec_connector_metadata=null) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] EngineCore encountered a fatal error. (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] Traceback (most recent call last): (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 835, in run_engine_core (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] engine_core.run_busy_loop() (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 862, in run_busy_loop (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] self._process_engine_step() (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 891, in _process_engine_step (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] outputs, model_executed = self.step_fn() (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 342, in step (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] model_output = future.result() (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] return self.__get_result() (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] raise self._exception (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/executor/uniproc_executor.py", line 79, in collective_rpc (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] result = run_method(self.driver_worker, method, args, kwargs) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/serial_utils.py", line 479, in run_method (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] return func(*args, **kwargs) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/worker_base.py", line 367, in execute_model (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] return self.worker.execute_model(scheduler_output, *args, **kwargs) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] return func(*args, **kwargs) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 563, in execute_model (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] output = self.model_runner.execute_model( (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] return func(*args, **kwargs) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 2655, in execute_model (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] self._update_states(scheduler_output) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 754, in _update_states (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] self._init_mrope_positions(req_state) (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 960, in _init_mrope_positions (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] model.get_mrope_input_positions( (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/paddleocr_vl.py", line 1234, in get_mrope_input_positions (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] t, h, w = image_grid_thw[image_index] (EngineCore_DP0 pid=1025) ERROR 11-28 13:11:39 [core.py:844] IndexError: list index out of range (EngineCore_DP0 pid=1025) Process EngineCore_DP0: (EngineCore_DP0 pid=1025) Traceback (most recent call last): (EngineCore_DP0 pid=1025) File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap (EngineCore_DP0 pid=1025) self.run() (EngineCore_DP0 pid=1025) File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run (EngineCore_DP0 pid=1025) self._target(*self._args, **self._kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 846, in run_engine_core (EngineCore_DP0 pid=1025) raise e (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 835, in run_engine_core (EngineCore_DP0 pid=1025) engine_core.run_busy_loop() (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 862, in run_busy_loop (EngineCore_DP0 pid=1025) self._process_engine_step() (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 891, in _process_engine_step (EngineCore_DP0 pid=1025) outputs, model_executed = self.step_fn() (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 342, in step (EngineCore_DP0 pid=1025) model_output = future.result() (EngineCore_DP0 pid=1025) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result (EngineCore_DP0 pid=1025) return self.__get_result() (EngineCore_DP0 pid=1025) File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result (EngineCore_DP0 pid=1025) raise self._exception (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/executor/uniproc_executor.py", line 79, in collective_rpc (EngineCore_DP0 pid=1025) result = run_method(self.driver_worker, method, args, kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/serial_utils.py", line 479, in run_method (EngineCore_DP0 pid=1025) return func(*args, **kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/worker_base.py", line 367, in execute_model (EngineCore_DP0 pid=1025) return self.worker.execute_model(scheduler_output, *args, **kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context (EngineCore_DP0 pid=1025) return func(*args, **kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 563, in execute_model (EngineCore_DP0 pid=1025) output = self.model_runner.execute_model( (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context (EngineCore_DP0 pid=1025) return func(*args, **kwargs) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 2655, in execute_model (EngineCore_DP0 pid=1025) self._update_states(scheduler_output) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 754, in _update_states (EngineCore_DP0 pid=1025) self._init_mrope_positions(req_state) (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 960, in _init_mrope_positions (EngineCore_DP0 pid=1025) model.get_mrope_input_positions( (EngineCore_DP0 pid=1025) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/paddleocr_vl.py", line 1234, in get_mrope_input_positions (EngineCore_DP0 pid=1025) t, h, w = image_grid_thw[image_index] (EngineCore_DP0 pid=1025) IndexError: list index out of range
🏃♂️ Environment (运行环境)
Package Version
aiohappyeyeballs 2.6.1 aiohttp 3.13.2 aiosignal 1.4.0 alembic 1.13.3 annotated-doc 0.0.4 annotated-types 0.7.0 anthropic 0.71.0 anyio 3.7.1 apache-tvm-ffi 0.1.1 argon2-cffi 25.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 astor 0.8.1 asttokens 3.0.0 async-generator 1.10 async-lru 2.0.5 async-timeout 5.0.1 attrs 25.3.0 babel 2.17.0 backcall 0.2.0 beautifulsoup4 4.14.0 blake3 1.0.7 bleach 6.2.0 cachetools 6.2.1 cbor2 5.7.0 certifi 2025.10.5 cffi 2.0.0 charset-normalizer 3.4.3 click 8.3.0 cloudpickle 3.1.2 comm 0.2.3 compressed-tensors 0.12.2 cuda-bindings 12.9.4 cuda-pathfinder 1.3.2 cuda-python 12.9.4 cupy-cuda12x 13.6.0 debugpy 1.8.16 decorator 5.2.1 defusedxml 0.7.1 depyf 0.20.0 dill 0.4.0 diskcache 5.6.3 distro 1.9.0 dnspython 2.8.0 docstring_parser 0.17.0 einops 0.8.1 email-validator 2.3.0 entrypoints 0.4 exceptiongroup 1.3.0 executing 2.2.1 fastapi 0.121.0 fastapi-cli 0.0.13 fastapi-cloud-cli 0.3.1 fastjsonschema 2.21.2 fastrlock 0.8.3 filelock 3.20.0 flashinfer-python 0.3.0 fqdn 1.5.1 frozenlist 1.8.0 fsspec 2024.10.0 gguf 0.17.1 greenlet 3.2.4 h11 0.16.0 hf-xet 1.1.7 httpcore 1.0.9 httptools 0.6.4 httpx 0.28.1 huggingface-hub 0.36.0 idna 3.10 interegular 0.3.3 ipykernel 6.17.0 ipython 8.12.0 ipython-genutils 0.2.0 ipywidgets 8.1.7 isoduration 20.11.0 jedi 0.19.2 Jinja2 3.1.6 jiter 0.12.0 jmespath 1.0.1 joblib 1.5.2 json5 0.12.0 jsonpointer 3.0.0 jsonschema 4.24.0 jsonschema-specifications 2025.9.1 jupyter 1.1.1 jupyter_client 7.4.4 jupyter-console 6.4.4 jupyter-contrib-core 0.4.2 jupyter-contrib-nbextensions 0.7.0 jupyter_core 4.11.2 jupyter-events 0.12.0 jupyter-highlight-selected-word 0.2.0 jupyter-lsp 2.2.5 jupyter_nbextensions_configurator 0.6.4 jupyter-server 1.21.0 jupyter_server_terminals 0.5.3 jupyterhub 1.0.0.dev0 jupyterlab 3.0.16 jupyterlab-pygments 0.2.2 jupyterlab_server 2.27.3 jupyterlab_widgets 3.0.15 lark 1.2.2 llguidance 1.3.0 llvmlite 0.44.0 lm-format-enforcer 0.11.3 loguru 0.7.3 lxml 6.0.0 Mako 1.3.10 markdown-it-py 4.0.0 MarkupSafe 3.0.3 matplotlib-inline 0.1.7 mdurl 0.1.2 mistral_common 1.8.5 mistune 2.0.5 model-hosting-container-standards 0.1.9 mpmath 1.3.0 msgpack 1.1.2 msgspec 0.19.0 multidict 6.7.0 nbclassic 0.5.6 nbclient 0.7.0 nbconvert 7.8.0 nbformat 5.10.3 nest-asyncio 1.6.0 networkx 3.4.2 ninja 1.13.0 notebook 6.4.0 notebook_shim 0.2.4 numba 0.61.2 numpy 1.26.4 nvidia-cublas-cu12 12.8.4.1 nvidia-cuda-cupti-cu12 12.8.90 nvidia-cuda-nvrtc-cu12 12.8.93 nvidia-cuda-runtime-cu12 12.8.90 nvidia-cudnn-cu12 9.10.2.21 nvidia-cudnn-frontend 1.16.0 nvidia-cufft-cu12 11.3.3.83 nvidia-cufile-cu12 1.13.1.3 nvidia-curand-cu12 10.3.9.90 nvidia-cusolver-cu12 11.7.3.90 nvidia-cusparse-cu12 12.5.8.93 nvidia-cusparselt-cu12 0.7.1 nvidia-cutlass-dsl 4.2.1 nvidia-ml-py 13.580.82 nvidia-nccl-cu12 2.27.5 nvidia-nvjitlink-cu12 12.8.93 nvidia-nvshmem-cu12 3.3.20 nvidia-nvtx-cu12 12.8.90 oauthlib 3.3.1 openai 2.7.1 openai-harmony 0.0.4 opencv-python-headless 4.11.0.86 outlines_core 0.2.11 overrides 7.7.0 packaging 25.0 pamela 1.2.0 pandas 2.2.3 pandocfilters 1.5.1 parso 0.8.5 partial-json-parser 0.2.1.1.post4 pexpect 4.9.0 pickleshare 0.7.5 pillow 12.0.0 pip 25.0.1 platformdirs 4.4.0 prometheus_client 0.23.1 prometheus-fastapi-instrumentator 7.1.0 prompt_toolkit 3.0.52 propcache 0.4.1 protobuf 6.33.0 psutil 7.0.0 ptyprocess 0.7.0 pure_eval 0.2.3 py-cpuinfo 9.0.0 pyarrow 18.0.0 pyavi 0.0.29 pybase64 1.4.2 pycountry 24.6.1 pycparser 2.22 pycryptodomex 3.21.0 pydantic 2.12.5 pydantic_core 2.41.5 pydantic-extra-types 2.10.6 Pygments 2.16.1 pynvml 13.0.1 pyorc 0.9.0 python-dateutil 2.9.0.post0 python-dotenv 1.2.1 python-json-logger 3.3.0 python-multipart 0.0.20 pytz 2025.2 PyYAML 6.0.3 pyzmq 27.1.0 ray 2.49.2 referencing 0.36.2 regex 2025.11.3 requests 2.32.5 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rich 14.2.0 rich-toolkit 0.15.1 rignore 0.7.0 rpds-py 0.20.1 safetensors 0.6.2 scikit-learn 1.5.2 scipy 1.15.3 Send2Trash 1.8.3 sentencepiece 0.2.1 sentry-sdk 2.43.0 setproctitle 1.3.7 setuptools 59.6.0 shellingham 1.5.4 six 1.17.0 sniffio 1.3.1 soupsieve 2.6 SQLAlchemy 2.0.43 stack-data 0.6.3 starlette 0.49.3 supervisor 4.2.5 sympy 1.14.0 tabulate 0.9.0 terminado 0.17.0 threadpoolctl 3.6.0 tiktoken 0.11.0 tinycss2 1.4.0 tokenizers 0.22.0 tomli 2.2.1 torch 2.9.0 torchaudio 2.9.0 torchvision 0.24.0 tornado 6.2 tqdm 4.67.1 traitlets 5.5.0 transformers 4.56.1 triton 3.5.0 typer 0.20.0 types-python-dateutil 2.9.0.20250708 typing_extensions 4.15.0 typing-inspection 0.4.2 tzdata 2025.2 uri-template 1.3.0 urllib3 2.5.0 uvicorn 0.38.0 uvloop 0.21.0 vllm 0.11.1 watchfiles 1.1.0 wcwidth 0.2.13 webcolors 24.11.1 webencodings 0.5.1 websocket-client 1.8.0 websockets 15.0.1 wheel 0.37.1 widgetsnbextension 4.0.14 xformers 0.0.33.post1 xgrammar 0.1.25 yarl 1.22.0
🌰 Minimal Reproducible Example (最小可复现问题的Demo)
from vllm import LLM, SamplingParams
llm = LLM(model='/data/pe-task/ai_storage_gcp_dsocr_test/20251118/PaddleOCR-VL-0.9B', trust_remote_code=True)
sampling_params = SamplingParams(temperature=0.6, top_p=0.9, max_tokens=20480)
model_path = "PaddlePaddle/PaddleOCR-VL"
image_path = "test.png"
task = "ocr" # Options: 'ocr' | 'table' | 'chart' | 'formula'
PROMPTS = {
"ocr": "OCR:",
"table": "Table Recognition:",
"formula": "Formula Recognition:",
"chart": "Chart Recognition:",
}
image = Image.open(image_path).convert("RGB")
messages = [
{"role": "user",
"content": [
{"type": "image", "image": image},
{"type": "text", "text": PROMPTS[task]},
]
}
]
# 生成回复
outputs = llm.chat(messages, sampling_params)
