Surprise SVDpp model fitting takes long time

Linux-3.10.0-957.1.3.el7.x86_64-x86_64-with-redhat-7.6-Maipo

Python 3.4.10 (default, Oct 4 2019, 19:14:13)

[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] surprise 1.1.1

Hi,

I am trying to run the SVDpp on a (user,item,feedback) data. I have around 2.5Million records and the fit method of the model is taking around 4 hours to complete. The code precisely looks as follows. Do you have any suggestions?

algo = SVDpp() trainset=data.build_full_trainset() #data has around 2.5M records algo.fit(trainset)

I was checking if i could alter any parameters to see if i can get that time down. The default n_epoch value is 20 and i was not sure if i can reduce it to reduce iterations and thereby the time fit method takes.

Is there anything that i can do to reduce the time taken to fit the model?. Any suggestion is welcome. Thanks in advance.

Sep 25 '20 03:09 ssrkaushik

reducing the number of iterations and the number of factors will result in faster training time. But that will potentially hurt RMSE

Sep 25 '20 03:09 NicolasHug

Ok, so with default parameters, the duration i had mentioned is expected(for the mentioned data size) is it? and there is no way to reduce the time it takes to fit the model?

Sep 25 '20 04:09 ssrkaushik

Sorry I don't know about the expected time

and there is no way to reduce the time it takes to fit the model?

I think I just answered that above

Sep 25 '20 12:09 NicolasHug