async-openai icon indicating copy to clipboard operation
async-openai copied to clipboard

fix chat completion image

Open Sherlock-Holo opened this issue 8 months ago • 0 comments

when marshal the detail field to '"detail": null', it will break vllm openai api server

3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] Error in preprocessing prompt inputs
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] Traceback (most recent call last):
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   File "/data/ai/pyenv/uv/vllm/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_chat.py", line 181, in create_chat_completion
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     ) = await self._preprocess_chat(
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   File "/data/ai/pyenv/uv/vllm/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_engine.py", line 388, in _preprocess_chat
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     conversation, mm_data_future = parse_chat_messages_futures(
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   File "/data/ai/pyenv/uv/vllm/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 951, in parse_chat_messages_futures
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     sub_messages = _parse_chat_message_content(
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   File "/data/ai/pyenv/uv/vllm/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 879, in _parse_chat_message_content
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     result = _parse_chat_message_content_parts(
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   File "/data/ai/pyenv/uv/vllm/lib/python3.12/site-packages/vllm/entrypoints/chat_utils.py", line 781, in _parse_chat_message_content_parts
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     for part in parts:
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]                 ^^^^^
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] pydantic_core._pydantic_core.ValidationError: 5 validation errors for ValidatorIterator
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] 1.typed-dict.text
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   Field required [type=missing, input_value={'type': 'image_url', 'im...Jf//Z', 'detail': None}}, input_type=dict]
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     For further information visit https://errors.pydantic.dev/2.10/v/missing
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] 1.typed-dict.type
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   Input should be 'text' [type=literal_error, input_value='image_url', input_type=str]
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     For further information visit https://errors.pydantic.dev/2.10/v/literal_error
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] 1.typed-dict.image_url.detail
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   Input should be 'auto', 'low' or 'high' [type=literal_error, input_value=None, input_type=NoneType]
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     For further information visit https://errors.pydantic.dev/2.10/v/literal_error
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] 1.typed-dict.input_audio
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   Field required [type=missing, input_value={'type': 'image_url', 'im...Jf//Z', 'detail': None}}, input_type=dict]
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     For further information visit https://errors.pydantic.dev/2.10/v/missing
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197] 1.typed-dict.type
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]   Input should be 'input_audio' [type=literal_error, input_value='image_url', input_type=str]
3月 07 16:43:07 mymachine llama-swap[2425068]: ERROR 03-07 16:43:07 serving_chat.py:197]     For further information visit https://errors.pydantic.dev/2.10/v/literal_error

maybe it is vllm problem, but omit the null field is a good idea

Sherlock-Holo avatar Mar 07 '25 09:03 Sherlock-Holo