[Question] Batch metrics and Top K Factorized do not correlate.

[ydennisy]

Hi!

I have a retrieval model which is showing the following behaviour:

• loss is reducing
• top k factorized is falling (degrading)
• top k batch metric is improving
• AUC (batch) is improving

Tweaking the model, I have seen this also swap around, but there have been a few configurations in which these metrics seem to not correlate.

I would really like to understand and learn under which conditions this is possible.

[patrickorlando]

Some things that come to mind:

• Ensure your batch size is reasonably large 1024+.
• Confirm that the output shapes from your query and candidate towers are (batch, query_dim). It's important the rank of these tensors is exactly 2.
• If you have relatively few candidates in your dataset, removing accidental hits becomes more important.