Giovanni Puccetti

@ykj467422034 sorry didn´t see your reply, so it repeats the same caption for different images or is generating several captions for one image? Also did you try and generate a...

Mmmh not sure, I asked about the random tensor to see if the model generates the same caption also in that case, if that is so, maybe fine-tuning didn´t go...

@ykj467422034 I think that with those changes you would still need to rerun the fine-tuning

hi @dsikka, can I ask you which generation type are you using? and how are you padding? There should be fixed length argument that the generator can use, these are...

Hi, @leifu1128 if I remember correctly it is related to this issue https://github.com/pytorch/pytorch/issues/71470

Hi, @ebsmothers the main reason is that this was mostly inspired by https://github.com/lucidrains/CoCa-pytorch/blob/main/coca_pytorch/coca_pytorch.py which uses parallel feedforward instead of the classic one both in self and cross attention.

| | dataset | model | acc1 | acc5 | |---:|:-----------|:--------------------|-------:|-------:| | 0 | imagenet1k | coca_ViT-B-32 | 0.636 | 0.881 | | 1 | imagenet1k | coca_ViT-B-32_fixed_cls_mask | 0.638...

> Re the remove MHA from attn pooler, I don't recall what the motivation for that was? It doesn't lool like it's doing anything that isn't supported by MHA at...

Hi @jemmyshin, I think there was an issue similar to this one that was fixed some time ago, any chance that you are using an older version? Otherwise this is...

Hi @jemmyshin, so indeed there is a little bug in some sense, however you can probably already do what you want, if I understand it without any changes in the...