Soonhwan-Kwon

> @Soonhwan-Kwon i am working on adding coco as a dataset so we can make evaluation automatically, should make the PR on a couple days unless you are doing it...

fix image_latents and image_tokens are calculated once instead of every seq_len. generate time reduced, 2.5967->1.8194, generate_beamsearch reduced(5 images) 53.1091->30.3831

@gpucce @rom1504 I'm middle of writing code for beam search and have a question that https://huggingface.co/laion/CoCa-ViT-B-32-laion2B-s13B-b90k/blob/main/epoch_95.pt is now outdated or not.

beam_search(num_beams=6) evaluation on coco(val2017): | | Blue_1 | Blue_2 | Blue_3 | Blue_4 | METEOR | ROUGE_L | CIDEr | SPICE | | ---- |---- | ---- | ---- |...

> @rwightman I'm definitely interested in pushing performance. As to merge readiness I think the main question is how do we ensure normal functionality isn't broken. Other stuff is more...

> @Soonhwan-Kwon I looked into adding beam_search (which is great as validation works a lot better!) and adding the `past_key_values` option seems to mean adding several changes, many in `transformer.py`,...

@gpucce removed changes related to past_key_values in https://github.com/mlfoundations/open_clip/pull/341