spaCy icon indicating copy to clipboard operation
spaCy copied to clipboard

Update cupy extras

Open adrianeboyd opened this issue 1 year ago • 2 comments

Description

  • Extend to v11
  • Add cupy-cuda11x and cupy-wheel
  • Update quickstart to use cupy-wheel for CUDA 10.2+

Types of change

Checklist

  • [x] I confirm that I have the right to submit this contribution under the project's MIT license.
  • [x] I ran the tests, and all new and existing tests passed.
  • [x] My changes don't require a change to the documentation, or if they do, I've added all required information.

adrianeboyd avatar Aug 08 '22 14:08 adrianeboyd

I'm not sure about the extra name cuda-wheel or the way the quickstart looks.

If you use cupy-wheel for all the existing CUDA versions in the quickstart (leaving them in the list), then it defaults to the first one (10.2), which works because all the commands are the same, but then it looks a bit weird as a default. But I think saying "10.2+" is going to confuse people. ???

adrianeboyd avatar Aug 08 '22 14:08 adrianeboyd

I don't really like cuda-wheel as the name, but we can't use plain cuda either because that's already used for source cupy. Maybe we can switch to maintaining the extras:

# keep cuda as source extra
cuda =
    cupy>=5.0.0b4,<12.0.0
# cupy as source extra
cupy =
    cupy>=5.0.0b4,<12.0.0
# cupy-wheel as wheel extra
cupy-wheel =
    cupy-wheel>=11.0.0,<12.0.0

We can keep the existing cuda* ones as is, but then just maintain / recommend these in the future?

adrianeboyd avatar Aug 09 '22 07:08 adrianeboyd

Looking at it again, I'm not sure cupy-wheel is stable enough that we want to recommend it yet across the board. It's not used in the main cupy install docs yet.

adrianeboyd avatar Sep 07 '22 07:09 adrianeboyd

For the record, this is the thread for status of cupy-wheel, which does seem to still be experimental:

https://github.com/cupy/cupy/issues/6688

It's not clearly stated, but it looks like it's supposed to work for CUDA 10.2 and not just 11+ - see the "supported packages" in the PR here.

It looks like there are some environments (like Docker) where there may be issues with it still, though it seems to be working fine for normal use.

polm avatar Sep 07 '22 09:09 polm

Hi, CuPy maintainer here. Technical issues discussed in https://github.com/cupy/cupy/issues/6688 has been resolved in v11 and I believe it's ready to use by downstream packages. Currently, it lacks documentation but we are going to do that in https://github.com/cupy/cupy/issues/7005.

I'd suggest naming the extras_require as cuda-autodetect. The primary benefit of the cupy-wheel package is that it automatically finds the CUDA version installed in the users' environment so that they don't have to know about their CUDA version and select which cupy-cudaXXX to use. For users having multiple CUDA versions installed on the same machine, however, it could be ambiguous whether CUDA picked by cupy-wheel is the one user actually want to use (cupy-wheel always try to find the latest CUDA version installed), so I think it's better to keep providing cuda11{01234567x} extras as well to cover such case.

kmaehashi avatar Sep 09 '22 04:09 kmaehashi

Thanks for the info! cuda-autodetect sounds like a good name to use.

adrianeboyd avatar Sep 09 '22 06:09 adrianeboyd