VLMEvalKit icon indicating copy to clipboard operation
VLMEvalKit copied to clipboard

[Model] add kosmos2

Open tackhwa opened this issue 1 year ago • 0 comments

  • tested on MME dataset c575b731fb648d324a4f1e511bd4080

observation: It seems that Kosmos2 does not pre-train well with strong knowledge, resulting in poor performance on datasets that require extensive knowledge (testes with MMBench, overall only get 0.013), but it performance is as expected while testing on other dataset which do not require as much knowledge:

  • BLINK dataset image

  • LLavaBench image

  • MMMU image

  • MMStar image

  • MMVet image

tackhwa avatar Sep 29 '24 19:09 tackhwa