baaivision
Invitation to support EMU2 in VLMEvalKit
Dear Authors, Congratulates to this great work. I'm Kenny from OpenCompass and we are building a comprehensive benchmark for multi-modality understanding. We would like to invite you to implement your great work -- EMU2 in our multi-modal evaluation benchmark VLMEvalKit. By implementing your work in this framework, you can easily utilize our 1-click solution to evaluation your VLM on tens of different benchmarks. Besides, according to your wishes, we can help include your model into the MMBench leaderboard and OpenCompass-MM leaderboard.
@Quan-Sun @yqy2001 Dear authors, will you consider this proposition? Actually we are glad to conduct this on our own, but we may be not sure that if the default prompts can reproduce your reported results.
Dear Kenny,
Thank you for your kind invitation! We are willing to add Emu2 to the VLMEvalKit multimodal benchmark and leaderboard.
Please establish contact with [email protected] and we will provide the prompt details for evaluation. We appreciate your help and effort :).
