models icon indicating copy to clipboard operation
models copied to clipboard

[Task] Implement ranking models integration tests

Open gabrielspmoreira opened this issue 3 years ago • 0 comments

Goals :soccer:

Create integration tests for ranking models (based on our research scripts) to check for

  • Stability of our API (unit tests) - by using advanced options (hparams) of the APIs that might break if changed
  • Accuracy and Throughput (integration tests) - Compute accuracy and throughput metrics for training and evaluation, and log properly so that it is possible to track over time if we have a drop in those metrics.

Starting point

  • Integration tests for retrieval models implemented in #537
  • Research scripts from the MTL research, including preprocessing of AliCCP and training/eval of ranking models for Single Task Learning (MLP, DLRM, DCN-v2, DeepFM, Wide&Deep) and Multi-Task Learning (MMoE, PLE)

gabrielspmoreira avatar Oct 04 '22 13:10 gabrielspmoreira