rapids_singlecell icon indicating copy to clipboard operation
rapids_singlecell copied to clipboard

Error moving adata.X to GPU

Open joseph-siefert opened this issue 5 months ago • 13 comments

Describe the bug When running either command adata.X = cpx.scipy.sparse.csr_matrix(adata.X) or rsc.get.anndata_to_GPU(adata), I get the following error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File <timed exec>:1

File [/mambaforge/envs/rapids_singlecell_24.06/lib/python3.11/site-packages/cupyx/scipy/sparse/_compressed.py:228](http://nruslj-cph510003:8888/mambaforge/envs/rapids_singlecell_24.06/lib/python3.11/site-packages/cupyx/scipy/sparse/_compressed.py#line=227), in _compressed_sparse_matrix.__init__(self, arg1, shape, dtype, copy)
    225 elif scipy_available and scipy.sparse.issparse(arg1):
    226     # Convert scipy.sparse to cupyx.scipy.sparse
    227     x = arg1.asformat(self.format)
--> 228     data = cupy.array(x.data)
    229     indices = cupy.array(x.indices, dtype='i')
    230     indptr = cupy.array(x.indptr, dtype='i')

File [/mambaforge/envs/rapids_singlecell_24.06/lib/python3.11/site-packages/cupy/_creation/from_data.py:53](http://nruslj-cph510003:8888/mambaforge/envs/rapids_singlecell_24.06/lib/python3.11/site-packages/cupy/_creation/from_data.py#line=52), in array(obj, dtype, copy, order, subok, ndmin, blocking)
      7 def array(obj, dtype=None, copy=True, order='K', subok=False, ndmin=0, *,
      8           blocking=False):
      9     """Creates an array on the current device.
     10 
     11     This function currently does not support the ``subok`` option.
   (...)
     51 
     52     """
---> 53     return _core.array(obj, dtype, copy, order, subok, ndmin, blocking)

File cupy[/_core/core.pyx:2408] in cupy._core.core.array()

File cupy[/_core/core.pyx:2435] in cupy._core.core.array()

File cupy[/_core/core.pyx:2578] in cupy._core.core._array_default()

File cupy[/_core/core.pyx:137] in cupy._core.core.ndarray.__new__()

File cupy[/_core/core.pyx:225] in cupy._core.core._ndarray_base._init()

File cupy[/cuda/memory.pyx:738] in cupy.cuda.memory.alloc()

TypeError: 'module' object is not callable

Steps/Code to reproduce bug

import os, sys, time
import cudf
import cupy as cp
import cupyx as cpx
import anndata
import scanpy as sc

from cuml.decomposition import PCA
from cuml.manifold import TSNE
from cuml.cluster import KMeans
from cuml.preprocessing import StandardScaler

import rapids_singlecell as rsc

import warnings
warnings.filterwarnings('ignore', 'Expected ')

import rmm
rmm.reinitialize(
    managed_memory=True, 
    pool_allocator=False,
    devices=0,
)

cp.cuda.set_allocator(rmm.allocators.cupy)

adata = sc.read('adata_file.h5ad')

The above works fine. Running either of the following gives the error adata.X = cpx.scipy.sparse.csr_matrix(adata.X) or rsc.get.anndata_to_GPU(adata)

Expected behavior Expected the command to be executed and adata.X be moved to GPU without error

Environment details (please complete the following information):

  • Environment location: Linux server
  • Linux Distro/Architecture: [Ubuntu 22.04 amd64]
  • GPU Model/Driver: [A100 and driver 535.129.03]
  • CUDA: [11.8 or 12.5]
  • Method of Rapids install: [conda, pip or from source]
    • Tested multiple methods of install, including: mamba env create -f conda/rsc_rapids_24.06.yml and mamba env create -f conda/rsc_rapids_24.08.yml with the yaml files from here: https://github.com/scverse/rapids_singlecell/tree/main/conda
  • Also tested installing rapids with: mamba create -n rapids-24.08 -c rapidsai -c conda-forge -c nvidia rapids=24.08 python=3.11 cuda-version=12.5 followed by: pip install 'rapids-singlecell[rapids12]' --extra-index-url=https://pypi.nvidia.com or: mamba create -n rapids-24.08 -c rapidsai -c conda-forge -c nvidia rapids=24.08 python=3.11 cuda-version=11.8 followed by: pip install 'rapids-singlecell[rapids11]' --extra-index-url=https://pypi.nvidia.com
  • All installations give same error

Additional context Add any other context about the problem here.

joseph-siefert avatar Sep 10 '24 18:09 joseph-siefert