KeyError: 'image_token_index'

[pavelgur]

I've spent some time trying to fine-tune Qwen/Qwen2.5-VL-7B-Instruct, and apparently I couldn't make it work. The Qwen/Qwen2.5-VL-7B-Instruct model yields an error

INFO:main:Training model 0%| | 0/2353 [00:00<?, ?it/s] Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "mlx_vlm/lora.py", line 178, in main(args) File "mlx_vlm/lora.py", line 99, in main dataset[i * args.batch_size : (i + 1) * args.batch_size] ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "mlx_vlm/trainer/trainer.py", line 89, in getitem image_token_index = self.config["image_token_index"] ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ KeyError: 'image_token_index'

so I decided to downgrade to Qwen/Qwen2-VL-7B-Instruct, and it worked. Maybe Qwen/Qwen2.5-VL-7B-Instruct requires additional support? Thank you.

[Blaizzy]

Maybe Qwen/Qwen2.5-VL-7B-Instruct requires additional support?

Hey @pavelgur

Thanks for reporting this.

It's a small bug in the MoldelConfig because Qwen2.5 uses image_token_id instead of the commonly used image_token_index.

I need to update the trained to support new formats of image tokens ids.

[Blaizzy]

Will fix in a upcoming release