lightfm
lightfm copied to clipboard
Make LightFM supports CuPy sparse matrices
First of all, thank you very much for that awesome package!
I think lightFM
could use CuPy
to enable GPU computation.
I have a lot of data and making predictions take long time.
I'm already using cupy
arrays to store the top-k recommended items, and I could decrease sorting the predictions using cp.argsort
by the half of time! That's a huge improvement.
Now I'm thinking that the sparse matrices calculations could be massive calculations too.
How about support lightFM to use cupy
sparse matrices?
https://docs.cupy.dev/en/stable/reference/sparse.html?highlight=sparse
I would reimplement the Cython part of the library using Numba or Jax to optimize for GPU and CPU at the same time.
Jax already has experimental sparse matrix support: https://jax.readthedocs.io/en/latest/jax.experimental.sparse.html