trax icon indicating copy to clipboard operation
trax copied to clipboard

Published 20 hours ago • verified publisher icon google

[BUG] Incorrect readings of crossentrophy loss in Trax 1.3.7

Open qiwen98 opened this issue 3 years ago • 1 comments

Description

The crossentrophy loss in Trax 1.3.7 gave strange reading. Is this normal? 1

The same code was running in Trax 1.3.6 with this result. 2

The loss layer and evaluation metrics used are both tl.CrossEntropyLoss().

qiwen98 avatar Dec 19 '20 12:12 qiwen98

Looks like tl.CrossEntropyLoss() is depreciated: https://trax-ml.readthedocs.io/en/latest/trax.layers.html#trax.layers.metrics.CrossEntropyLoss However, it looks like tl.WeightedCategoryCrossEntropy() might also have a memory leak (at least on TPU) so hold off on switching to trax 1.3.7 for now?

JEF1056 avatar Jan 18 '21 06:01 JEF1056