Haodong Duan
Haodong Duan
Hi , @sachit-menon Everything is zero-shot now. We do not officially support few-shot since only few models support that.
Definitely Yes! Welcome to implement your new dataset VL-ICL in VLMEvalKit. After implemented, we can help with the evaluation of API models. BTW, you can check our latest refactor PR...
Hi, @mary-0830 , 1. 有计划 2. 目前我们已支持 API 评估，请参考 GPT-4v, GeminiProVision 等样例 3. 后续有安排，约在一两周后
@Quan-Sun @yqy2001 Dear authors, will you consider this proposition? Actually we are glad to conduct this on our own, but we may be not sure that if the default prompts...
Hi, @lj163ucas , We are still evaluating those VLMs on MME and SEEDBench, the results will be available in 1 / 2 weeks. The data on the previous page are...
Hi, @lj163ucas , the evaluation of those benchmarks are now supported in [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) (including the evaluation results). Besides, we would add a multi-modal leaderboard in the following several days to...
公公又式式
Which conda version are you using? If your conda version is earlier than the version listed in the repo README, please update and retry.
Dear chenscottus, I'm not familiar with onnx so may not be able to answer this question. Other users may have such experience.
It's fine I think. You just need to use the weighted summation of scores for different modalities (joint, bone, etc.) as the fused prediction.