I-BERT icon indicating copy to clipboard operation
I-BERT copied to clipboard
verified publisher icon kssteven418

Fix NaN values in IntLayerNorm

Open AmosHason opened this issue 3 years ago • 1 comments

Add an epsilon to std_int in the forward pass of IntLayerNorm to prevent NaN values due to multiplication of 0 by infinity. The same result can alternatively be achieved by calling torch.nan_to_num(y_int).

AmosHason avatar Jan 29 '23 05:01 AmosHason

Fixes #31.

AmosHason avatar Jan 29 '23 05:01 AmosHason