Multimodal-Object-Detection-via-Probabilistic-Ensembling icon indicating copy to clipboard operation
Multimodal-Object-Detection-via-Probabilistic-Ensembling copied to clipboard

How to use demo_probEn.py for the KAIST dataset?

Open s1mple-game opened this issue 1 year ago • 1 comments

Thank you for your hard work and contributions! I have successfully reproduced the results in Table 1 of your paper using your pre-trained models KAIST_rgb_only.pth and KAIST_thermal_only.pth. However, when using demo/FLIR/demo_probEn.py to fuse the results of RGB and Thermal with ProbEn and s-avg, I obtained slightly different results:

MR_all = 9.48, MR_day = 11.34 , MR_night = 5.72

But in your paper, it is :

MR_all = 8.67, MR_day = 10.27, MR_night = 5.41

The only modification I made was that I use "bayesian_fusion" but not "bayesian_fusion_multiclass",and I input "match_score" instead of "match_prob", like: final_score= bayesian_fusion(np.asarray(match_score))

Did I miss something? Because for KAIST I set the num_classes to 1, match_prob should be equivalent to match_score.

s1mple-game avatar Mar 24 '23 15:03 s1mple-game

Hi, when I used KAIST_thermal_only.pth to evaluate the LAMR of baseline-Thermal, I obtained the results below: MR_all: 19.45 MR_day: 25.30 MR_night: 7.76 but in the paper, they are: MR_all: 18.99 MR_day: 24.59 MR_night: 7.76 Do you obtain the same results as the paper shown?

fannyzhouha avatar Aug 25 '23 02:08 fannyzhouha