addons icon indicating copy to clipboard operation
addons copied to clipboard

GroupNorm + FP16 instability

Open shkarupa-alex opened this issue 4 years ago • 1 comments

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Colab
  • TensorFlow version and how it was installed (source or binary): 2.6.0
  • TensorFlow-Addons version and how it was installed (source or binary): 0.14.0
  • Python version: 3
  • Is GPU used? (yes/no): no

Describe the bug In some cases GroupNormalization layer generates Inf/Nan values with mixed_fp16 policy.

Code to reproduce the issue https://colab.research.google.com/drive/17zqRy30dbOcJpWZfOwZK3UFovIJZCuqb?usp=sharing

shkarupa-alex avatar Aug 25 '21 07:08 shkarupa-alex

/cc @smokrow

bhack avatar Sep 23 '21 15:09 bhack