Switch to conda-forge PyTorch, install only conda-forge libs (pyg-lib, torch-scatter)

[alexbarghi-nv]

[bdice]

With this PR, do we still need to skip tests on CUDA 12? I think there should be packages available for CUDA 12 on the new channels. If not, we should identify what is missing.

 RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────┐
|    skipping cugraph_dgl pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────┘


RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────┐
|    skipping cugraph_pyg pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────┘


RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────────────┐
|    skipping cugraph-equivariant pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────────────┘

[alexbarghi-nv]

With this PR, do we still need to skip tests on CUDA 12? I think there should be packages available for CUDA 12 on the new channels. If not, we should identify what is missing.

 RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────┐
|    skipping cugraph_dgl pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────┘


RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────┐
|    skipping cugraph_pyg pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────┘


RAPIDS logger » [04/17/24 20:09:23]
┌─────────────────────────────────────────────────────────┐
|    skipping cugraph-equivariant pytest on CUDA!=11.8    |
└─────────────────────────────────────────────────────────┘

I just reenabled CUDA 12 testing. Let's see what happens.

[alexbarghi-nv]

CUDA 12 testing still seems to be blocked because our CI is using CUDA 12.2 but there are only CUDA 12.1 packages for PyG, DGL and PyTorch available on conda.

[jakirkham]

Think the packages should be compatible with CUDA 12.2. Let's give this another try

[alexbarghi-nv]

@jakirkham both the PyG and DGL tests for CUDA 12.2 failed because of the CUDA version incompatibility.

[alexbarghi-nv]

Mamba wasn't able to install the right packages because we had a conflict between packages for CUDA 12.2 and CUDA 12.1. I'm not sure how we can resolve this in conda.

[bdice]

@alexbarghi-nv @jakirkham Perhaps this can help (copying from an internal discussion). You can just install CUDA 12.1 from conda-forge to align with PyTorch's CUDA version.

Try this. PyTorch only has CUDA 12.1 builds on the pytorch channel, so you probably need to match to that. RAPIDS will work with CUDA 12.1, even though the install page only lists 12.0 and 12.2.

mamba create -n pytorch-rapids-cuda-12.1 rapids pytorch torchvision torchaudio pytorch-cuda=12.1 cuda-version=12.1 -c rapidsai -c pytorch -c conda-forge -c nvidia

I verified that both cudf and torch can be imported and used:

import cudf
import torch

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
arr = torch.ones(100, device=device)
print(arr)

print(cudf.Series(arr))

[alexbarghi-nv]

Closing this; I think we need to wait until the repository migration, which will pin the RAPIDS version and resolve that problem.