topaz icon indicating copy to clipboard operation
topaz copied to clipboard

RuntimeWarning: overflow encountered in exp in training

Open huwjenkins opened this issue 4 years ago • 2 comments

Hi,

I see the following message printed to stderr when training. RuntimeWarning: overflow encountered in exp. This is reproducible with Topaz 0.2.4 on the test dataset.

Using numpy.seterr identifies this occurs here for scores < -88.7228 as then np.exp(-score) overflows float32.

https://github.com/tbepler/topaz/blob/11f7ecbdfedfd4820199d43571059de61748c17d/topaz/commands/train.py#L539

Is this an issue? Thanks.

huwjenkins avatar Aug 27 '20 08:08 huwjenkins

Thanks for pointing this out. It should be fixed, but won't cause any problems other than the annoying warning.

tbepler avatar Aug 28 '20 14:08 tbepler

Thanks for confirming it doesn't cause any problems! Fortunately the warning is only printed the first time it happens.

huwjenkins avatar Aug 28 '20 14:08 huwjenkins