DeepLearningExamples icon indicating copy to clipboard operation
DeepLearningExamples copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

[ConvNet/PyTorch] SyncBatchNorm not Used in ConvNet ImageNet Classification

Open YuchuanTian opened this issue 1 year ago • 0 comments

Related to ConvNet/PyTorch (e.g. GNMT/PyTorch or FasterTransformer/All)

Describe the bug SyncBatchNorm is not used in the script https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/Classification/ConvNets. Is this a potential bug when DDP is activated? I make this guess because I did not see any relevant code in this repo that resolves the out-sync problem between different GPUs.

YuchuanTian avatar Jul 22 '22 08:07 YuchuanTian