Captioning off-the-shelf

[shrutijpalaskar]

Hello,

I am using pre-trained VL-T5 to generate captions for Flickr30K images off-the-shelf i.e. without any finetuning. I modified the captioning scripts to predict directly. I observe very short captions through, almost like noun phrases. I am including some examples below. I have played with the '--gen_max_length' and '--num_beams' parameters but I still get very short outputs. Do you have any ideas why this may be happening? Or any suggestions for how to potentially generate longer captions?

Thank you in advance! Shruti

purple shirt
cutting cake
smiling
large group of people
skier

[j-min]

It's probably because the pretraining objective for text generation (span prediction) always involves short target text. I guess zero-shot captioning might now work well. You would need to tune the parameters at least slightly, through few-shot or full fine-tuning.