flox icon indicating copy to clipboard operation
flox copied to clipboard

CuPy support

Open dcherian opened this issue 1 year ago • 4 comments

  1. We should test with numpy-groupies. CuPy provides bincount, https://github.com/cupy/cupy/issues/7561
  2. We'd have to avoid factorizing with Pandas unfortunately and use np.searchsorted or np.digitize; or use CuDF?

dcherian avatar Aug 02 '22 16:08 dcherian

Turns out scatter_* are a repacement for ufunc.at not ufunc.reduceat. The former does not take an axis kwarg so we can't use it for now.

cupy 12 might have more ufunc support, so let's wait and see

dcherian avatar Aug 16 '22 03:08 dcherian

Hi, CuPy maintainer here! We've created a dedicated tracker for ufunc features: https://github.com/cupy/cupy/issues/7082 Do you need reduceat for specific ufuncs or all ufuncs?

kmaehashi avatar Oct 04 '22 05:10 kmaehashi

Thanks for checking in @kmaehashi

Technically we could use

  • add.reduceat, multiply.reduceat, maximum.reduceat, and minimum.reduceat (with engine="flox") and
  • multiply.at, maximum.at, and minimum.at (for engine="numpy" which uses numpy_groupies)

The second one is the default now, supporting .at would be most helpful

For reduceat, add.reduceat would be the most useful since that lets us do sum, mean, var and std

dcherian avatar Oct 04 '22 16:10 dcherian

Thanks! Sounds it aligns with our v12 target 😄 cc/ @asi1024

kmaehashi avatar Oct 05 '22 04:10 kmaehashi