Support cuTENSOR wheel

[leofang]

This issue tracks two aspects of the cuTENSOR wheel:

1. The installer (cupyx.tools.install_library) can take advantage of the cuTENSOR wheel
   • ideally we wanna do pip install cutensor so that pip is made aware of it
   • potential issue:
     • the cutensor wheel is very large (~300 MB as of 1.6.0) as it contains 3 versions (10.2, 11.0, 11), we are internally discussing if we can split it
2. The runtime loader (cupy._environment._preload_library('cutensor')) should first check from site-packages:
   • This issue is more urgent as we (cuQuantum) noticed wheel users can end up loading two copies of libcutensor.so, causing potentially hard-to-debug problems.

@kmaehashi I'd like to get the second aspect addressed asap.

[kmaehashi]

1. The installer (cupyx.tools.install_library) can take advantage of the cuTENSOR wheel

   • ideally we wanna do pip install cutensor so that pip is made aware of it

   • potential issue:

     • the cutensor wheel is very large (~300 MB as of 1.6.0) as it contains 3 versions (10.2, 11.0, 11), we are internally discussing if we can split it

Sounds good. (note: cuTENSOR for Windows seem unavailable on PyPI)

2. The runtime loader (cupy._environment._preload_library('cutensor')) should first check from site-packages:

   • This issue is more urgent as we (cuQuantum) noticed wheel users can end up loading two copies of libcutensor.so, causing potentially hard-to-debug problems.

I was just thinking of a similar. I think it's OK to do this.

[leofang]

(note: cuTENSOR for Windows seem unavailable on PyPI)

Thanks for bringing this up 🙂 I'll bring this up for internal discussion. cuQuantum is currently the only consumer of cuTENSOR wheels so Windows was not yet prioritized.

[kmaehashi]

It's done in #7025