Assertion Failure in CUDA Indexing: srcIndex < srcSelectDimSize

[tobiashaab]

Hello everyone!

When trying to run the model LLaVA-CoT locally with the command

python VLMEvalKit/run.py --data AI2D_TEST --model LLaVA-CoT_local --verbose --mode infer --reuse

I am encountering the following error:

/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1500: indexSelectSmallIndex: block: [6,0,0], thread: [14,0,0] Assertion `srcIndex < srcSelectDimSize` failed.

I have defined LLaVA-CoT_local in the file config.py the following way:

"LLaVA-CoT_local": partial(llama_vision, model_path="Llama-3.2V-11B-cot"),

PyTorch versions:

• torch==2.7.1
• torchvision==0.22.1

Any help would be appreciated!