recommenders icon indicating copy to clipboard operation
recommenders copied to clipboard

tfrs.metrics.FactorizedTopK with tensorflow_macos and tensorflow_metal

Open Tractorage opened this issue 2 years ago • 0 comments

Metrics computed by FactorizedTopK seems very off when running on M1 Max. Tried it on the "Quickstart" movielens example with Adam(0.005) and received the following evaluation results:

{'factorized_top_k/top_5_categorical_accuracy': 0.007550000213086605,
 'factorized_top_k/top_10_categorical_accuracy': 0.018650000914931297,
 'factorized_top_k/top_15_categorical_accuracy': 0.03215000033378601,
 'factorized_top_k/top_20_categorical_accuracy': 0.2814500033855438,
 'factorized_top_k/top_25_categorical_accuracy': 0.31675001978874207,
 'factorized_top_k/top_30_categorical_accuracy': 0.7000000476837158,
 'factorized_top_k/top_35_categorical_accuracy': 0.6884000301361084,
 'factorized_top_k/top_40_categorical_accuracy': 0.6511000394821167,
 'loss': 28395.984375,
 'regularization_loss': 0,
 'total_loss': 28395.984375}
tensorflow-macos==2.10.0
tensorflow-recommenders ==0.7.2
tensorflow-metal==0.6.0

Tractorage avatar Nov 03 '22 12:11 Tractorage