SLIM-recommendation icon indicating copy to clipboard operation
SLIM-recommendation copied to clipboard

Published 20 hours ago • verified publisher icon SSSxCCC

efficient

Open XqWang3 opened this issue 4 years ago • 5 comments

When I am running the slim on movielen20m datasets with 14W users or so, the time is too long for me to calculate the recommendation results ! About two days. So could you provide some suggestions for me to accelerate the training. Thank you for any answer.

XqWang3 avatar Jul 16 '20 02:07 XqWang3

I just limits the max_worker = 5 due to the matter of the memory.

XqWang3 avatar Jul 16 '20 02:07 XqWang3

movielen20m is a huge dataset, you should make sure that your RAM is big enough. Too long running time might be caused by out of memory. Also you can try the tensorflow implemetation version of SLIM code wrriten by me, you can find the link in readme.

SSSxCCC avatar Jul 17 '20 05:07 SSSxCCC

Thank you for your answer. Indeed, I have tried the tensorflow version of SLIM, but the same problem has happened. By the way, the total space of my RAM is 188G or so.

XqWang3 avatar Jul 17 '20 05:07 XqWang3

movielen20m is a huge dataset, you should make sure that your RAM is big enough. Too long running time might be caused by out of memory. Also you can try the tensorflow implemetation version of SLIM code wrriten by me, you can find the link in readme.

Have you ever played the tf version of SLIM on ML-20M ? The dataset is also mentioned in your code.

XqWang3 avatar Jul 17 '20 06:07 XqWang3

My computer has only 48GB RAM. Out of memory happened when I run SLIM on ML-20M so that I didn't run it successfully. It really cost too much RAM. Further optimization to the code may solve the efficient issue.

SSSxCCC avatar Jul 17 '20 14:07 SSSxCCC