llava_arch.py image_sizes missing. (Mistral LLava 1.6)[Usage]

[huwprosser]

Issue: When running the following command with the 1.6 mistral model I get an image_size error. I DID manage to get it working by manually setting the image size I'm inputting inside llava_arch.py

Command:

    --model-path liuhaotian/llava-v1.6-mistral-7b \
    --image-file "image.png" --device mps

Log:

/LLaVA/llava/model/llava_arch.py", line 173, in prepare_inputs_labels_for_multimodal num_patch_width, num_patch_height = get_anyres_image_grid_shape(image_sizes[image_idx], self.config.image_grid_pinpoints, self.get_vision_tower().config.image_size) TypeError: 'NoneType' object is not subscriptable

[storuky]

same here

[heltrix]

Same here, looks like the call to model.generate in cli.py is not passing the image_sizes variable. Adding the line: imagesize=image.size below the load_image() call, and the line: image_sizes = [imagesize], to the model.generate() call fixed it for me.

https://github.com/haotian-liu/LLaVA/pull/1038

[JoeySalmons]

Same here, looks like the call to model.generate in cli.py is not passing the image_sizes variable. Adding the line: imagesize=image.size below the load_image() call, and the line: image_sizes = [imagesize], to the model.generate() call fixed it for me.

#1038

This seems to also apply to other files, such as run_llava.py, except it needs to work with one or more images. Something like this should work for run_llava.py (I have tested this with a single image but not multiple images.)

def load_images(image_files): out = [] sizes = [] for image_file in image_files: image = load_image(image_file) out.append(image) sizes.append(image.size) return out, sizes

modify the function call: images, imagesizes = load_images(image_files)

and add a line to the model.generate call: image_sizes=imagesizes,

[Silviase]

I have encountered the same error, and this solution worked for me, thank you 👍

[huwprosser]

Same here, looks like the call to model.generate in cli.py is not passing the image_sizes variable. Adding the line: imagesize=image.size below the load_image() call, and the line: image_sizes = [imagesize], to the model.generate() call fixed it for me.

#1038

This worked for me, thank you everyone! 🚀