Not able to reproduce BEVDet4d result

[LigZhong]

Hi, I tried to reproduce some results of BEVDet and BEVDet4d and encountered some problems. Does any met the same problem.

My reproduction of BEVDet-sttiny result basically matched the official mAP and NDS though the losses are in general lower than those released in the log. But for BEVDet4d-sttiny, my result are 0.437 and 0.3157 for NDS and mAP, which are lower than expected. And some other experiments related to 4d are also problematic. I loaded the official ckpt of BEVDet4d and the result are the same as the reported ones. The training configuration seems to be OK. Any help?

[HuangJunJie2017]

@LigZhong have you pulled and used the latest code?

[LigZhong]

My code may not be up to date. I will rerun the experiments and see if it works. Thanks

[HuangJunJie2017]

@LigZhong check this commit~ https://github.com/HuangJunJie2017/BEVDet/commit/196fed98adddf1d8b5989b5eb3028af5067157f2

[HuangJunJie2017]

@LigZhong If you use another version before this commit, you just need to test the trained model with the script of tools/test.py to get a more precise result

[LigZhong]

@LigZhong If you use another version before this commit, you just need to test the trained model with the script of tools/test.py to get a more precise result

Is this the only difference? I did change that option but the result does not differ too much. I am rerunning the experiment now. Hopefully it can be solved>-<

[HuangJunJie2017]

@LigZhong report the full results with all ma*e?