addons
addons copied to clipboard
GroupNorm + FP16 instability
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Colab
- TensorFlow version and how it was installed (source or binary): 2.6.0
- TensorFlow-Addons version and how it was installed (source or binary): 0.14.0
- Python version: 3
- Is GPU used? (yes/no): no
Describe the bug In some cases GroupNormalization layer generates Inf/Nan values with mixed_fp16 policy.
Code to reproduce the issue https://colab.research.google.com/drive/17zqRy30dbOcJpWZfOwZK3UFovIJZCuqb?usp=sharing
/cc @smokrow