llama-models icon indicating copy to clipboard operation
llama-models copied to clipboard

Can llama3.2 vision accept no image?

Open hessaAlawwad opened this issue 1 year ago • 1 comments

Hello,

thank you for thegreat work. I have been trying to expermint with the model and see how it works.

My question is: Can I use llama3.2 vision to cover cases where there is no image?

I tried set the value of the image variable to None or even delete it but I alawys get: ValueError: Invalid input type. Must be a single image, a list of images, or a list of batches of images. OR ValueError: No image were provided, but there are image tokens in the prompt

hessaAlawwad avatar Oct 10 '24 17:10 hessaAlawwad

Can you show your code?

ashwinb avatar Oct 10 '24 21:10 ashwinb

like in this:

question = row['Question']
options = row['AnswerChoices']

message = (f"Answer the question with the correct option letter. "
               f"Question: {question}\nOptions: {options}")

messages = [
        {"role": "user", "content": [
            {"type": "text", "text": message}
        ]}
    ]

  
input_text = processor.apply_chat_template(messages, add_generation_prompt=True)

inputs = processor(
        input_text,
        add_special_tokens=False,
        return_tensors="pt"
    ).to(model.device)

output = model.generate(**inputs, max_new_tokens=30)
response = processor.decode(output[0])

return response

I have also tried sending None for the image but got error

hessaAlawwad avatar Oct 19 '24 11:10 hessaAlawwad

@ashwinb

hessaAlawwad avatar Oct 30 '24 05:10 hessaAlawwad

The processor method is expecting the images attribute to be set with value None . Link

Try the following and if it doesn't work please re-open the issue.

inputs = processor(
        images=None,
        text=input_text,
        add_special_tokens=False,
        return_tensors="pt"
    ).to(model.device)

varunfb avatar Nov 12 '24 05:11 varunfb