sglang fix: remove dependency on latest transformers impl

Motivation

Current is_valid_list_of_images import in qwen2_5_vl_config.py relies on latest transformers, leading to an import issue shown here

Modifications

qwen2_5_vl_config.py , is_valid_list_of_images copied from transformers

Checklist

[x] Format your code according to the Code Formatting with Pre-Commit.
[ ] Add unit tests as outlined in the Running Unit Tests.
[ ] Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
[ ] Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
[ ] For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
[ ] Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

Feb 17 '25 11:02 mickqian

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

Feb 17 '25 20:02 vhain

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

Please use the transformers version suggested by sglang, thanks

Feb 18 '25 00:02 mickqian

I will give credit to Chen-0210 also. @Chen-0210

Feb 18 '25 16:02 zhaochenyang20

Thanks for the fix. but I am seeing another error:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/sglang/launch_server.py", line 6, in <module>
    from sglang.srt.entrypoints.http_server import launch_server
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/http_server.py", line 41, in <module>
    from sglang.srt.entrypoints.engine import _launch_subprocesses
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/entrypoints/engine.py", line 36, in <module>
    from sglang.srt.managers.data_parallel_controller import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/data_parallel_controller.py", line 27, in <module>
    from sglang.srt.managers.io_struct import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/io_struct.py", line 24, in <module>
    from sglang.srt.managers.schedule_batch import BaseFinishReason
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/managers/schedule_batch.py", line 42, in <module>
    from sglang.srt.configs.model_config import ModelConfig
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/__init__.py", line 4, in <module>
    from sglang.srt.configs.qwen2_5_vl_config import (
  File "/usr/local/lib/python3.10/dist-packages/sglang/srt/configs/qwen2_5_vl_config.py", line 1005, in <module>
    AutoImageProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLImageProcessor)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/image_processing_auto.py", line 628, in register
    IMAGE_PROCESSOR_MAPPING.register(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 833, in register
    raise ValueError(f"'{key}' is already used by a Transformers model.")
ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model.

when using latest transformers:

transformers==4.49.0

I've tried with a workaround as this PR with transformers==4.49.0: https://github.com/sgl-project/sglang/pull/3792

Feb 23 '25 04:02 panpan0000

@panpan0000 I don't think you can work with transformers==4.49.0. Use transformers==4.48.3 please.

Feb 23 '25 06:02 zhaochenyang20

Trying to understand the nature of the issue, looks like in new transformers version if it already has the Qwen2VL image processors.

ValueError: '<class 'sglang.srt.configs.qwen2_5_vl_config.Qwen2_5_VLConfig'>' is already used by a Transformers model. ('Qwen2VLImageProcessor', 'Qwen2VLImageProcessorFast')

so the question is why do we have our own versions here in sglang. If it's expected, we can just register and accept that exist_ok?

+AutoImageProcessor.register(Qwen2_5_VLConfig, None, Qwen2_5_VLImageProcessor, None, exist_ok=True)
+AutoProcessor.register(Qwen2_5_VLConfig, Qwen2_5_VLProcessor, exist_ok=True)

Mar 17 '25 22:03 yinghai

Thanks @yinghai. The latest transformers has compatibility issue with sglang, that's why we have to keep a version of configs / processors. However the modification you mentioned is also ok. By the time the issues above is resolved here, all the associated codes will be removed.

Mar 18 '25 01:03 mickqian