ValueError when Loading Qwen2-VL Model with Liger Kernel

[rahatarinasir]

trafficstars

🐛 Describe the bug

I'm encountering a ValueError when trying to load the Qwen2-VL model using the AutoLigerKernelForCausalLM class from the Liger Kernel. The error message indicates an unrecognized configuration class for this model.

Reproduce

Clone the repository: bash

git clone https://github.com/linkedin/Liger-Kernel.git cd Liger-Kernel pip install -e .

Install necessary packages: python

from huggingface_hub import login hf_token = input("Enter your Hugging Face token: ") login(token=hf_token) pip install torch pip install triton # >= 2.3.0 pip install liger-kernel pip install liger-kernel-nightly pip install git+https://github.com/huggingface/transformers@21fac7abba2a37fae86106f87fcf9974fd1e3830

Attempt to load the model: python

from transformers import Qwen2VLConfig, Qwen2VLForConditionalGeneration from liger_kernel.transformers import AutoLigerKernelForCausalLM

config = Qwen2VLConfig.from_pretrained("Qwen/Qwen2-VL-2B-Instruct") model = AutoLigerKernelForCausalLM.from_pretrained("Qwen/Qwen2-VL-2B-Instruct", config=config)

Error Message angelscript

ValueError: Unrecognized configuration class <class 'transformers.models.qwen2_vl.configuration_qwen2_vl.Qwen2VLConfig'> for this kind of AutoModel: AutoLigerKernelForCausalLM. Model type should be one of [list of model types].

It seems that the AutoLigerKernelForCausalLM class does not recognize the Qwen2VLConfig. Is there a workaround or an update planned to support this model? Any guidance would be greatly appreciated!

Versions

Environment Report:

Operating System: Linux-6.1.85+-x86_64-with-glibc2.35 Python version: 3.10.12 PyTorch version: 2.4.0+cu121 CUDA version: 12.1 Triton version: 3.0.0 Transformers version: 4.45.0.dev0