BatchNorm behavior during inference

[slala2121]

This is bit more detailed but I wanted to check if the batch norm layer of the computation graph (1st picture below) matches the one you find when importing the model.

When zeroing in on the model graph loaded under inference mode, it seems that the batch norm doesn't use the running_mean and running_var during forward pass (see 1st graph below; I also checked by computing expected output and it didn't match).

But when I construct a simpler net, batch norm uses the running_mean and running_var during forward pass (see 2nd graph; I also checked by computing expected output).

I'm wondering if this is could be an error related to FusedBatchNorm vs FusedBatchNormV3; upon importing the model, it uses FusedBN yet when constructing a new model, it used FusedBNV3. I'm not sure what causes this difference in the type of class used to construct the BN layer.

Thanks.

[mikevoets]

Hi @slala2121, thanks for your question! Sorry for my extremely late reply.

Can you first please let me know what Python and Tensorflow version you're running this with? And give me some instructions on how to get this graph please?