blog icon indicating copy to clipboard operation
blog copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

[New blog post] Unified multimodal large model evaluation, accelerating multimodal intelligence emergence

Open kcz358 opened this issue 1 year ago • 7 comments

Hi @lewtun , this is our blog for the lmms-eval. Could you help us check the article and see whether there are something that can be added for example user experience or how to add a new model? Also, you might also want to add your names in the author list.

Thank you!

This blog introduces a new evaluation pipeline for large vision language model. Building upon lm-evaluation-harness, this framework has been improved and expanded to provide a unified interface for defining models, datasets, and evaluation metrics, offering a one-stop, efficient solution for evaluating multimodal models (LMMs). We hope that through this framework, we can collectively drive the iteration cycle of multimodal models and promote their broader application in academia and industry.

kcz358 avatar Apr 15 '24 02:04 kcz358

Hi @lewtun , thank you for your feedback.

I have uploaded the thumbnail picture and fixed several problems in the blog. Could you help us check if there are any more problems to fix in this article?

When we finalize the English version of the article, we will also help to translate everything into Chinese and put it into /blog/zh

Thank you!

kcz358 avatar Apr 20 '24 05:04 kcz358

thanks a lot for the blog post! I'll give this a spin 😊

merveenoyan avatar Apr 24 '24 15:04 merveenoyan

Hi @pcuenca @merveenoyan , thank you for your kind feedback.

I have tried to fix most of the issue in the comments and the image source issue. May I kindly ask for a review for this version and I will try to update the date in _blog.yml before release.

kcz358 avatar Apr 25 '24 13:04 kcz358

Thanks for iterating @kcz358 ! Would you mind resolving the merge conflicts and then we should be pretty good to go!

lewtun avatar May 01 '24 14:05 lewtun

Hi @lewtun , I have merged the main branch and added the Chinese version of the blog. I have also updated the date in _blog.yml

kcz358 avatar May 02 '24 05:05 kcz358

Hi @lewtun , sorry for pinning you again. Do you think we are able to merge for current version?

kcz358 avatar May 16 '24 06:05 kcz358

@pcuenca I resolved the merge conflicts - ok if we merge this? (Feel free to do so if you agree)

lewtun avatar May 30 '24 14:05 lewtun