Giovanni Puccetti

> @gpucce @rom1504 I'm middle of writing code for beam search and have a question that https://huggingface.co/laion/CoCa-ViT-B-32-laion2B-s13B-b90k/blob/main/epoch_95.pt is now outdated or not. @Soonhwan-Kwon it is still the only one there...

@rom1504 I think adding https://github.com/sks3i/pycocoevalcap to the dependencies would make adding cider and other captioning metrics very easy, is it fine to add it? otherwise I can add similar code...

Initial super naive generation evaluation on coco(val2017): | | Bleu_1 | Bleu_2 | Bleu_3 | Bleu_4 | METEOR | ROUGE_L | CIDEr | SPICE | |---:|---------:|----------:|----------:|-----------:|----------:|----------:|---------:|----------:| | 0 | 0.254451...

@rom1504 @rwightman For info I will continue working on this until it is at a good point as far as it is needed (hoping my effort gets the whole thing...

checking the FLAVA PR I list here the main differences to choose the way to go: - [ ] outputting dicts is an option in COCA while it is enforced...

@iejMac @Soonhwan-Kwon @rom1504 @rwightman With 6 epoch of fine-tuning on mscoco, with the very first model we trained and with beam_search I get | index | Bleu_1 | Bleu_2 |...

> looking better! for fine-tune, how much extra code is it? I feel fine-tune falls closer to a train focused repo than benchmarking? I did it on open_clip creating a...

@Soonhwan-Kwon I looked into adding beam_search (which is great as validation works a lot better!) and adding the `past_key_values` option seems to mean adding several changes, many in `transformer.py`, if...

@fxmarty asking as I can´t really get glue as good ad in the paper, if you have run also other glue tasks, did you have to apply similar changes also...

Great! I'd like to add 2 things. An output like the one pROC gives in R and the Delong test for roc differences significance. That said, I'm not sure I...