[QST] Questions about topk_model.evaluate()

[ZhanqiuHu]

❓ Questions & Help

Details

I was looking at this tutorial and came across these two lines of code. I'm curious about what evaluation and sampling methods are used to compute the topk metrics, and how evaluation batch size and shuffling affect the computed metrics.

eval_loader = mm.Loader(valid, batch_size=1024).map(mm.ToTarget(schema, "item_id"))
metrics = topk_model.evaluate(eval_loader, return_dict=True)

Thank!

[sararb]

The top-k model uses a brute force evaluation where each user query scores against the whole catalog of items and by default shuffling is set to False for eval and inference mode.

[rnyak]

closing due to low activity, please reopen if you have further questions.