rapids_singlecell
rapids_singlecell copied to clipboard
scverse
Error when running rsc.pp.highly_variable_genes
Hi, rapids_singlecell raised an error when running rsc.pp.highly_variable_genes. Could you please take a look at it？Below are the error information. Thanks in advance!
Traceback (most recent call last):
File "
I never seen this error. I also don't think that this is a bug within rapids-singlecell. It looks more like a cupy bug.
elif issparse(X):
if not require_cf:
return True
if X.has_canonical_format:
return True
This is a quick fix for the affected lines. Just copy this in
I modified the code according to your suggestion, replacing
elif issparse(X):
if X.has_canonical_format or not require_cf:
return True
else:
X.sort_indices()
X.sum_duplicates(
with
elif issparse(X):
if not require_cf:
return True
if X.has_canonical_format:
return True
However, it raised another error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/lustre2/zeminz_pkuhpc/lingxinnan/00.setting/source_scripts/scVis_ToolKit.py", line 450, in rsc_harmony_pipel
ine
rsc.pp.highly_variable_genes(adata, n_top_genes=HVGs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/rapids_single
cell/preprocessing/_hvg.py", line 159, in highly_variable_genes
_check_gpu_X(X)
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/rapids_single
cell/preprocessing/_utils.py", line 92, in _check_gpu_X
if not require_cf:
^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupyx/scipy/s
parse/_compressed.py", line 676, in __get_has_canonical_format
self._has_canonical_format = bool(is_canonical.all())
^^^^^^^^^^^^^^^^^^
File "cupy/_core/core.pyx", line 1169, in cupy._core.core._ndarray_base.all
File "cupy/_core/core.pyx", line 1171, in cupy._core.core._ndarray_base.all
File "cupy/_core/_routines_logic.pyx", line 8, in cupy._core._routines_logic._ndarray_all
File "cupy/_core/_reduction.pyx", line 618, in cupy._core._reduction._SimpleReductionKernel.__call__
File "cupy/_core/_reduction.pyx", line 370, in cupy._core._reduction._AbstractReductionKernel._call
File "cupy/_core/_cub_reduction.pyx", line 689, in cupy._core._cub_reduction._try_to_call_cub_reduction
File "cupy/_core/_cub_reduction.pyx", line 526, in cupy._core._cub_reduction._launch_cub [1/1946]
File "cupy/_core/_cub_reduction.pyx", line 461, in cupy._core._cub_reduction._cub_two_pass_launch
File "cupy/_util.pyx", line 64, in cupy._util.memoize.decorator.ret
File "cupy/_core/_cub_reduction.pyx", line 240, in cupy._core._cub_reduction._SimpleCubReductionKernel_get_cached_$unction
File "cupy/_core/_cub_reduction.pyx", line 223, in cupy._core._cub_reduction._create_cub_reduction_function
File "cupy/_core/core.pyx", line 2254, in cupy._core.core.compile_with_cache
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/co$piler.py", line 484, in _compile_module_with_cache
return _compile_with_cache_cuda(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/co$piler.py", line 562, in _compile_with_cache_cuda
ptx, mapping = compile_using_nvrtc(
^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/co$piler.py", line 319, in compile_using_nvrtc
return _compile(source, options, cu_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 290, in _compile
options, headers, include_names = _jitify_prep(
^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 233, in _jitify_prep
jitify._init_module()
File "cupy/cuda/jitify.pyx", line 220, in cupy.cuda.jitify._init_module
File "cupy/cuda/jitify.pyx", line 244, in cupy.cuda.jitify._init_module
File "cupy/cuda/jitify.pyx", line 217, in cupy.cuda.jitify._init_cupy_headers
File "cupy/cuda/jitify.pyx", line 200, in cupy.cuda.jitify._init_cupy_headers_from_scratch
File "cupy/cuda/jitify.pyx", line 275, in cupy.cuda.jitify.jitify
RuntimeError: Runtime compilation failed
By the way, the cupy version is 13.1.0 and the rapids version is 24.04.00
I have never seen this. I can also not reproduce it at the moment I might be able to look into it on Monday. I would also like to know what I the rsc version you are using and what is the type of your matrix? It would also be helpful if you used the template and gave a reproducible code snippet.
The rsc version is 0.10.2 I used the pbmc3k dataset from sc.datasets.pbmc3k_processed().
>>> adata
AnnData object with n_obs × n_vars = 2638 × 13714
obs: 'n_genes', 'percent_mito', 'n_counts', 'louvain'
var: 'n_cells'
uns: 'draw_graph', 'louvain', 'louvain_colors', 'neighbors', 'pca', 'rank_genes_groups'
obsm: 'X_draw_graph_fr', 'X_pca', 'X_tsne', 'X_umap'
obsp: 'connectivities', 'distances'
>>> adata.X
<2638x13714 sparse matrix of type '<class 'numpy.float32'>'
with 2238732 stored elements in Compressed Sparse Row format>
It raised error earlier running rsc.pp.filter_genes(adata, min_count=5)
>>> import cupy as cp
>>> import rapids_singlecell as rsc
>>>
>>> import rmm
>>> from rmm.allocators.cupy import rmm_cupy_allocator
>>> rmm.reinitialize(
... managed_memory=True,
... pool_allocator=False,
... )
>>> cp.cuda.set_allocator(rmm_cupy_allocator)
>>> if input_format=='CPU':
... rsc.get.anndata_to_GPU(adata)
... elif input_format=='GPU':
... pass
>>> rsc.pp.filter_genes(adata, min_count=5)
Running `calculate_qc_metrics` for 'n_cells_by_counts','total_counts','mean_counts' or 'pct_dropout_by_counts'
/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/compiler.py
:233: PerformanceWarning: Jitify is performing a one-time only warm-up to populate the persistent cache, this may tak
e a few seconds and will be improved in a future release...
jitify._init_module()
rsc.pp.highly_variable_genes(adata, n_top_genes=HVGs)^[[D^H^H^H^H1000Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/rapids_single
cell/preprocessing/_simple.py", line 311, in filter_genes
calculate_qc_metrics(adata=adata, log1p=False)
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/rapids_single
cell/preprocessing/_simple.py", line 73, in calculate_qc_metrics
_check_gpu_X(X)
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/rapids_single
cell/preprocessing/_utils.py", line 92, in _check_gpu_X
if not require_cf:
^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupyx/scipy/s
parse/_compressed.py", line 676, in __get_has_canonical_format
self._has_canonical_format = bool(is_canonical.all())
^^^^^^^^^^^^^^^^^^
File "cupy/_core/core.pyx", line 1169, in cupy._core.core._ndarray_base.all
File "cupy/_core/core.pyx", line 1171, in cupy._core.core._ndarray_base.all
File "cupy/_core/_routines_logic.pyx", line 8, in cupy._core._routines_logic._ndarray_all
File "cupy/_core/_reduction.pyx", line 618, in cupy._core._reduction._SimpleReductionKernel.__call__
File "cupy/_core/_reduction.pyx", line 370, in cupy._core._reduction._AbstractReductionKernel._call
File "cupy/_core/_cub_reduction.pyx", line 689, in cupy._core._cub_reduction._try_to_call_cub_reduction
File "cupy/_core/_cub_reduction.pyx", line 526, in cupy._core._cub_reduction._launch_cub
File "cupy/_core/_cub_reduction.pyx", line 461, in cupy._core._cub_reduction._cub_two_pass_launch
File "cupy/_util.pyx", line 64, in cupy._util.memoize.decorator.ret
File "cupy/_core/_cub_reduction.pyx", line 240, in cupy._core._cub_reduction._SimpleCubReductionKernel_ge[122/1961]
unction
File "cupy/_core/_cub_reduction.pyx", line 223, in cupy._core._cub_reduction._create_cub_reduction_function
File "cupy/_core/core.pyx", line 2254, in cupy._core.core.compile_with_cache
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 484, in _compile_module_with_cache
return _compile_with_cache_cuda(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 562, in _compile_with_cache_cuda
ptx, mapping = compile_using_nvrtc(
^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 319, in compile_using_nvrtc
return _compile(source, options, cu_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 290, in _compile
options, headers, include_names = _jitify_prep(
^^^^^^^^^^^^^
File "/lustre1/zeminz_pkuhpc/lingxinnan/anaconda3/envs/rapids_singlecell/lib/python3.11/site-packages/cupy/cuda/com
piler.py", line 233, in _jitify_prep
jitify._init_module()
File "cupy/cuda/jitify.pyx", line 220, in cupy.cuda.jitify._init_module
File "cupy/cuda/jitify.pyx", line 244, in cupy.cuda.jitify._init_module
File "cupy/cuda/jitify.pyx", line 217, in cupy.cuda.jitify._init_cupy_headers
File "cupy/cuda/jitify.pyx", line 200, in cupy.cuda.jitify._init_cupy_headers_from_scratch
File "cupy/cuda/jitify.pyx", line 275, in cupy.cuda.jitify.jitify
RuntimeError: Runtime compilation failed
I can't reproduce the error. I looks like an error with cupy to me. I think you have an issue there.