Zeros in moving_mean and moving_var in the batchnorm

[WassimBouatay]

I am converting my model from the checkpoint file to an Onnx

I set the parameter is_train = False in the Class batch_norm of the file tf2onnx\onnx_opset\nn.py so that my batchnorm takes the moving_mean and moving_variance it learned during the training. My first problem, is that in my onnx file, as you can see in the screenshot below: the moving_mean and moving_variance is all zeros. This is probably due to the graph optimization operations, because the input of the batch_norm node doesn't contain moving_mean and moving_variance, although they exist in the graph before the optimization.

My second problem, is that I don't understand why the name of the mean/var is the one of layer_108 instead of layer_1.

I am using:

• Tensorflow Version: 1.15
• Python version: 3.6
• Opset 9

I am available to provide more information if you need. Thank you for your help

[TomWildenhain-Microsoft]

Hi @WassimBouatay, can you try again using the latest tf2onnx? pip install --upgrade tf2onnx I think we might have fixed a similar issue a while back.

If that fails please upload the model and I'll take a look. Also what opset are you using for conversion?

[WassimBouatay]

Thank you @TomWildenhain-Microsoft . I will try that tomorrow, and I am using opset 9

[WassimBouatay]

@TomWildenhain-Microsoft I am sorry for the late answer. I was busy working on something else. I tried the latest tf2onnx, but I still had the same problem. Again, I had to set is_training = False, but the mean and variance are 0. If I don't set is_training to false, then a mean and a variance branches are added in the graph to calculate them from the input (which is not what I want).

[WassimBouatay]

@TomWildenhain-Microsoft

I have also noticed that this problem happens only for batchnorms that follows seperable_conv2d.

For instance in the first conv (which is a normal conv2d) of the following image, the batch_norm layer was successfully merged with the convolution (the ouptut in the second image shows that the batch normalization was done correctly) which is not the case for the second convolution which is a seperable_conv2d