CUDA 13 in conda-forge

[h-vetinari]

The next cuda major version was released.

I'm sure @conda-forge/cuda will bring this up as soon as time permits.🙏 (Update: see tracking issue for cuda toolkit components: https://github.com/conda-forge/cuda-feedstock/issues/85)

I'm opening this issue for tracking purposes and visibility. One thing that stands out to me is the upper limit on the driver version in Table 2:

CTK Version	Driver Min	Driver Max
13.x	>= 580	N/A
12.x	>= 525	< 580
11.x	>= 450	< 525

If I'm reading this correctly we might want to make an announcement that users shouldn't update their CUDA driver past 580 for the time being, if they want to keep our current CUDA-enabled packages.

Also noteworthy is that CUDA 13 drops support for Maxwell, Pascal and Volta.

[bdice]

Users should be safe to update to driver 580 while using CUDA 12.x toolkits / libraries. Environments with CUDA 12 libraries should continue to work with updated drivers.

Table 2 of the link you shared is about minor version compatibility. Since CUDA 13 is a different major version, "minor version compatibility" is not the compatibility model used, it's "backwards compatibility" which has fewer requirements and is thus essentially automatic for users. (I agree this is a bit confusingly written in the page you linked.)

I recommend referencing this document about compatibility rules: https://docs.nvidia.com/deploy/cuda-compatibility/

Specifically, this section:

CUDA Compatibility guarantees allow for upgrading only certain components:

• Backwards compatibility ensures that a newer NVIDIA driver can be used with an older CUDA Toolkit. This is implicit and most simple way of doing upgrades.
• Minor version and forward compatibility ensure that an older NVIDIA driver can be used with a newer CUDA Toolkit.

CUDA driver (libcuda.so)	Compatibility type	Requirements
Newer than the CUDA runtime	Backwards compatibility	None
Older than the CUDA runtime, but same major version of the CUDA runtime	Minor Version Compatibility	No PTX (requires SASS), NVCC target architecture required
Older than the major version of the CUDA runtime	Forward Compatibility	Extra CUDA compatibility package

[bdice]

Also https://github.com/conda-forge/cuda-feedstock/issues/85 is a tracking issue for the CUDA 13 bringup. 😄

[jakirkham]

Also https://github.com/conda-forge/cuda-feedstock/issues/85 is a tracking issue for the CUDA 13 bringup. 😄

For clarity that is tracking the CUDA Toolkit update to CUDA 13.0

Packages outside that scope are best tracked in different issues

[h-vetinari]

Thanks for the clarification @bdice. Great to see that this is already in progress! 💪

Do you already have plans w.r.t. timing for a CUDA 13.x migration? As soon as all the components are available, or rather sometime later?

[bdice]

We are making progress on rolling out the component libraries for now. I think we'll want a migration soon, we'll probably want to discuss that once all the components are available.

[billysuh7]

We are making progress on rolling out the component libraries for now. I think we'll want a migration soon, we'll probably want to discuss that once all the components are available.

I second this - all CTK13 components are available now. @bdice @h-vetinari would you help by creating a migrator in conda-forge-pinning feedstock, as we are ready to migrate non-CTK packages. I am told these are good examples to go by: this and this. Thanks!

Also just as an FYI I have created https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/395 to track cross compilation issue using CTK13

[h-vetinari]

I think it would be good to wind down the CUDA 12.9 migration first? 🤔

We can start 13.0 independently if people want to get started earlier, but then it should at least use wait_for_migrators:. We'll also need to extend the ordering in the cuda 12.9 & 11.8 migrators.

Before start a migration though, we should fix things like https://github.com/conda-forge/conda-forge-ci-setup-feedstock/issues/395, and demonstrate that this works end-to-end on 2-3 feedstocks (e.g. by using a draft migrator with use_local: true). We should probably also double-check any other fixes from the rollout of 12.x that need to be adapted for 13.x.

Also, did the glibc lower bound increase with CUDA 13.0? Is glibc 2.17 still expected to work?

I am told these are good examples to go by: this and this.

Various pieces have changed since then. A closer reference would be the current 12.9 migrator.

[bdice]

Thanks! You shared lots of helpful info. I haven't had a chance to dig into this but wanted to answer one of the questions.

Also, did the glibc lower bound increase with CUDA 13.0? Is glibc 2.17 still expected to work?

The lowest glibc version supported by CUDA 13 is 2.28 (equivalent to RHEL / CentOS / Rocky / Alma 8). Source: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/

[carterbox]

I've started prototyping a migrator in this PR: https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/7653

[bdice]

Migration progress is here: https://conda-forge.org/status/migration/?name=cuda130