LMDrive icon indicating copy to clipboard operation
LMDrive copied to clipboard

Cannot reproduce the reported performance

Open MYZhangwww opened this issue 1 year ago • 1 comments

Hi, thanks for your excellent work and code release! The closed-loop experiments are so exciting! However, I tried to reproduce the experimental results using the official code and model weights(LLaVA-v1.5-7B) on the LangAuto Long benchmark on an RTX 4090. The CARLA version was 0.9.10.1, but the results I obtained were lower than those reported in the paper. I got DS=25.044, RC = 36.526, but the results reported in the paper are: DS=36.2, RC=46.5. I find previous issues also encountered this problem, is there anything wrong in the released code?

MYZhangwww avatar May 15 '24 13:05 MYZhangwww

Please refer to: https://github.com/opendilab/LMDrive/issues/56

deepcs233 avatar Jul 22 '24 16:07 deepcs233