lightfm icon indicating copy to clipboard operation
lightfm copied to clipboard

Make LightFM supports CuPy sparse matrices

Open igorkf opened this issue 3 years ago • 1 comments

First of all, thank you very much for that awesome package!

I think lightFM could use CuPy to enable GPU computation.
I have a lot of data and making predictions take long time.
I'm already using cupy arrays to store the top-k recommended items, and I could decrease sorting the predictions using cp.argsort by the half of time! That's a huge improvement.

Now I'm thinking that the sparse matrices calculations could be massive calculations too.
How about support lightFM to use cupy sparse matrices?
https://docs.cupy.dev/en/stable/reference/sparse.html?highlight=sparse

igorkf avatar Nov 16 '20 14:11 igorkf

I would reimplement the Cython part of the library using Numba or Jax to optimize for GPU and CPU at the same time.

Jax already has experimental sparse matrix support: https://jax.readthedocs.io/en/latest/jax.experimental.sparse.html

dbalabka avatar Jul 28 '22 14:07 dbalabka