NiftyNet
NiftyNet copied to clipboard
default value of `use_local_stats` in batch norm layer
It seems that in many window-based segmentation/regression use cases, either settingis_training=True
or setting use_local_stats=True
in the batch normalisation layer at inference time leads to better performance than those with NiftyNet's default batch norm options.
Therefore this ticket proposes to add a use_local_stats
option in the config file, and make use_local_stats=True
as the default for segmentation application with batch norm.
Specifically, in the current implementation (v0.4.0) the default values are:
- training:
is_trainign=True
use_local_stats=True
- inference:
is_training=False
use_local_stats=True
which uses running averages from training at inference time, following Ioffe and Szegedy (2015) https://arxiv.org/abs/1502.03167.
The proposed change will use these values by default:
- training:
is_trainign=True
use_local_stats=True
- inference:
is_training=False
use_local_stats=True
which will effectively follow Ulyanov et al. (2017) https://arxiv.org/abs/1607.08022.
This ticket is related to https://github.com/NifTK/NiftyNet/issues/280, https://github.com/NifTK/NiftyNet/issues/285, https://github.com/NifTK/NiftyNet/pull/282.
thoughts?
Probably to be addressed with #282