The specific value of the metric？

[yabuke]

The value of 2.463 for the CIREr metric in Table 10 of the paper, is it converted? Is it 2.463 or 0.02463?

[ChonghaoSima]

it's converted to 0-100 (the range of value).

[yabuke]

so, the 2.463 in paper is corresponding to 0.02463 calculated by the evaluation.py?

[ChonghaoSima]

Yes, in the evaluation.py we don't convert the range of the value

---

发件人: yabuke @.> 发送时间: 2024年4月23日 16:10 收件人: OpenDriveLab/DriveLM @.> 抄送: Sima, Chonghao @.>; Comment @.> 主题: Re: [OpenDriveLab/DriveLM] The specific value of the metric？ (Issue #81)

---- External Email: Use caution with attachments, links, or sharing data ----

so, the 2.463 in paper is corresponding to 0.02463 calculated by the evaluation.py?

― Reply to this email directly, view it on GitHubhttps://github.com/OpenDriveLab/DriveLM/issues/81#issuecomment-2071686195, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AM6RGLVIUZ5UACHARZUATMTY6YJOLAVCNFSM6AAAAABGSVD2LGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZRGY4DMMJZGU. You are receiving this because you commented.Message ID: @.***>

[yabuke]

I tested the evaluation.py using only one sample with correct answer, and got the value of CIDEr is 0.0. I don't know much about this metric. Is this no problem?

[yabuke]

[yuxr-smile]

Did you solve the problem？I met the same problem.