fundamentalvision
Has anyone successfully reproduced the experimental results?
I recently tried to reproduce the experimental results of bevformerv2 (bevformerv2-r50-t1-base-24ep.py), but I the performance was much worse.
I also could not reproduce the results of BEVFormer v2. For bevformerv2-r50-t1-base-24ep I got 0.391 NDS and 0.301 mAP and for bevformerv2-r50-t1-24ep 0.340 NDS and 0.249 mAP. Both results are much lower (~ 5-10 mAP) than with the provided checkpoints. @zhiqi-li @htian01 Could you please provide us with more information on how to reproduce v2? That would be very nice.
Firstly whether the provided ckpt is able to get exactly same performance in your env? The training log are also provided, with the same setting, check the whether are exactly similiar. It should be with the same seed. Diff your setting and the log also.
Some common fauiler migh be.
- you load wrong pretrain weight.
- your version of env are different. for example, mmdet3d, the lib is too complex to compatible. Thr mmdet3d also involved with data process.
Otherwise, More information, log for example might be helpful.
