xtuner icon indicating copy to clipboard operation
xtuner copied to clipboard

xtuner mmbench on xtuner/ lava-Llama 3-8b-v1_1-hf

Open Yanllan opened this issue 9 months ago • 1 comments

I have a model of lava-Llama 3-8b-v1_1-hf format How do I run xtuner mmbench on a model in xtuner/ lava-Llama 3-8b-v1_1-hf format? Now it can only accept models in the xtuner/ lava-Llama -3-8b-v1_1 format.

Yanllan avatar May 10 '24 10:05 Yanllan

@Yanllan Sorry, lava-Llama 3-8b-v1_1-hf format evaluation is not currently supported.

xtuner/ lava-Llama -3-8b-v1_1_hf is a version derived from xtuner/ lava-Llama -3-8b-v1_1, primarily for user convenience. If you would like to conduct an evaluation, you can directly use xtuner/ lava-Llama -3-8b-v1_1 as both versions have identical performance.

hhaAndroid avatar May 10 '24 10:05 hhaAndroid