DeepInteraction icon indicating copy to clipboard operation
DeepInteraction copied to clipboard

MMPI's unfair comparing with detr problem

Open AndyYuan96 opened this issue 1 year ago • 0 comments

Hi, Zeyu, thanks for sharing the code, I just have one question about your comparing between MMPI and detr, as you use ROI feature, I think your MMPI is more similar to a two stage way, so does it fair to comparing two stage with detr? according to centerpoint's author's multimodal version paper MVP(https://arxiv.org/pdf/2111.06881.pdf) Table 3, a very simple two stage can give a improvement of 1.1 mAP and 0.8 NDS, and according to your paper, you use MMPI for Lidar and image, you can give a improvement of 1.3 mAP and 1.0 NDS, which is better than centerpoint's two stage, but you use 5 encoder layer, which means you use roi feature refine many times, if you only refine one time, with 2 decoders, the performance is mAP 69.5 NDS 72.3, your improvement is 0.9 mAP, 0.7 NDS,which is lower than two stage only refine one time。 So I think that it’s more convince that you have a fair compare with centerpoint‘s two stage to claim that your MMPI is more useful, rather than comparing with detr。

AndyYuan96 avatar Mar 20 '23 15:03 AndyYuan96