can't use CBN to train a resnet model from scratch？ask for help

[nkyle04]

I was using a resent18 model from torchvision，and try to replace all bn by CBN. But I can't train it at all, the accuracy is always very little, such as 0.01%

I also use an pre trained resnet18 model that has been trained for 40 epochs, and continue train with CBN replaced for some epochs, and the accuracy is also dropped immediately.

Is there any details I had missed？Please give me some help, Thanks.