InstanceRefer icon indicating copy to clipboard operation
InstanceRefer copied to clipboard

Published 20 hours ago • verified publisher icon CurryYuan

About evaluating on Referit3D

Open ghost opened this issue 4 years ago • 3 comments
trafficstars

Hi, thanks for your wonderful work. Will the evaluation code on Referit3D be released? Thanks!

ghost avatar May 13 '21 06:05 ghost

@hsuanlyh1997 Yes, we will release codes for Referit3D in the future.

yanx27 avatar May 15 '21 06:05 yanx27

Hi @yanx27, thanks for your awesome work. Could you please release it soon? I tried to reproduce results for referit3d using your code but I am getting the following results:

overall | overall | [email protected]: 0.3151 overall | overall | [email protected]: 0.2525

Probably I am doing something wrong, but anyway a trained checkpoint for SR3D and NR3D would be really helpful. Thanks!

ayushjain1144 avatar Sep 26 '21 17:09 ayushjain1144

Hi @ayushjain1144 , the Acc@mIoU is only used in ScanRefer. For referit3d, we use GT instance as input and evaluate the grounding accuracy solely. Please refer to their paper for details. I think the result will be much higher if you follow this setting.

CurryYuan avatar Sep 27 '21 04:09 CurryYuan