VILA icon indicating copy to clipboard operation
VILA copied to clipboard

Published 20 hours ago • verified publisher icon NVlabs

OpenVLM leaderboard

Open oroojlooy opened this issue 1 year ago • 3 comments

Do you have any plan to get involved in OpenVLM leaderboard? https://huggingface.co/spaces/opencompass/open_vlm_leaderboard I think that needs some efforts from your side, but given the performance of VILA provides you good visibility.

oroojlooy avatar Jun 21 '24 18:06 oroojlooy

@oroojlooy sure, we would love to. Our models and inference examples are already released. Any other efforts we need to get involved in the benchmark?

Lyken17 avatar Jul 03 '24 21:07 Lyken17

@oroojlooy as a followup, how can we submit vila to the leaderboard?

Lyken17 avatar Aug 14 '24 16:08 Lyken17

Hi @Lyken17 Looks like I missed your first message. Just the follow the instruction as mentioned in the page:

OpenVLM Leaderboard only includes open-source VLMs or API models that are publicly available. To add your own model to the leaderboard, please create a PR in [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) to support your VLM and then we will help with the evaluation and updating the leaderboard. For any questions or concerns, please feel free to contact us at [opencompass, duanhaodong]@pjlab.org.cn.

oroojlooy avatar Aug 14 '24 16:08 oroojlooy