tensorflow
About using label smoothing
Hello,
while training my retrieval model I used categorical cross entropy with label smoothing as my loss function in order not to punish high scores on (user, item) pairs that might be false negatives. When trying to combine it with the candidate_ids to remove accidental hits I got very high loss values.
By looking at your code I realized that this is due to the fact that RemoveAccidentalHits sets these "hits" to the lowest possible value (as log(x) tends to minus infinity as x tends to 0). However, when using label smoothing the way to remove these accidental hits is not by setting these values to be close to minus infinity but to the logarithm of label_smoothing/n_samples
I made a quick fix by myself but maybe you should consider this question.
@josealbertof did you have any performance improvement for label smoothing? as i believe it particularly suitable for recommendations (we don't know the GT)
