BEVFormer icon indicating copy to clipboard operation
BEVFormer copied to clipboard

Published 20 hours ago • verified publisher icon fundamentalvision

Questions about SyncBatchNorm

Open tdzdog opened this issue 2 years ago • 0 comments

I noticed that batchsize=1 is applied for each gpu while the syncbatchnorm is not applied. Wouldn't this lead to inaccurate parameter updates for the BN layer? I have tried to train bevformer-small on 4 RTX 3090 with batchsize=2 per gpu. I got NDS=49.0, which is higher than the reported 47.9. I'm wondering if this results from SyncBN.

tdzdog avatar Dec 08 '22 13:12 tdzdog