implicit icon indicating copy to clipboard operation
implicit copied to clipboard

MemoryError

Open ArtKip opened this issue 3 years ago • 5 comments

Hello, thanks for a great library!

I faced a problem, that BayesianPersonalizedRanking model can't allocate memory for fitting data neither using gpu nor cpu mode.

While using gpu mode error seems strange, that it is not connected to gpu memory:

Screenshot 2022-02-01 at 09 25 43

While using cpu mode, I have another error:

Screenshot 2022-02-01 at 09 32 40

Each of these errors are produced on the same data. AlternatingLeastSquares fits ok on this data, but it is slower than bpr model.

ArtKip avatar Feb 01 '22 06:02 ArtKip

How much CPU memory do you have available? The BPR model and ALS model should use a similar amount of RAM, so its a little surprising to me that the BPR model fails and the ALS model succeeds.

Also, the GPU side looks like its using an old version of this library - does this still happen with the latest version? The new GPU bpr code doesn't allocate CPU memory , and will instead create the factors directly on the GPU.

benfred avatar Feb 01 '22 18:02 benfred

I use 0.4.8 version, I'll try the new version now. After restarting notebook I was able to fit BPR model on cpu, so probably it was some issue with Jupyter notebook.

ArtKip avatar Feb 01 '22 19:02 ArtKip

Let me know how it goes with the new version! Just as a heads up, there are a bunch of breaking api changes with the 0.5.0 https://github.com/benfred/implicit/issues/481 - but the flip side is that there are a bunch of performance improvements that should make the hassle worth it.

benfred avatar Feb 01 '22 19:02 benfred

After update to newer version I face the same problem, but with bigger number of factors.

Screenshot 2022-02-02 at 00 04 03

For update I used:

conda update -c conda-forge implicit
conda update -c conda-forge implicit-proc

is it a correct way?

ArtKip avatar Feb 01 '22 21:02 ArtKip

Also I would like to ask some side question. If I want to find audience for particular item, is it a correct way to fit item_user_data (instead of user_item_data) and use recommend function for this item?

ArtKip avatar Feb 01 '22 21:02 ArtKip