inplace_abn for transformer

[17dacheng]

dear author, the transformer use the layer norm in stead of batch norm, is it possible to apply inplace abn to transformer-based models? or is there any way to lower those models' gpu memory? thanks.