Ross Wightman

@qfmy it's not normal and it does not occur for me. Are you sure there isn't an issue on your setup loading the pretrained weights? corrupt file download?

If you happened to have enabled stochastic depth or dropouts, you need model.eval() after creation, just updated the README

@sandeepmukh I think a few things wrong for this .... first, update to main branch. Then, I think this is needed in CocaModel to replace current vocab_size logic btw text...

@MengqingCao hmm, yeah this needs fixing. One q though as I'm not intimately familiar with the gen code. Does the implementation here support multiple sentences in a batch? If yes,...

Moving this to discussions now for reference.

@alvaro-stylesage the coca embeds are a bit wrong... https://github.com/mlfoundations/open_clip/issues/458#issuecomment-1457281651 it 'works' but it's not 100% correct

will close this until bigger picture re #434 is resolved at some point

@work4cs a SigLIP model using one of the SigLIP model configs (e.g. https://github.com/mlfoundations/open_clip/blob/main/src/open_clip/model_configs/ViT-B-16-SigLIP-256.json) will not have this issue, but if you train a 'non-siglip' model by enabling siglip, yes you...

@airogachev not sure why you think this is an issue or bug vs the model being too large for the batch size? siglip is improved scaling over clip but it...

@gpucce thanks, I saw this, will take a closer look soon