UMAP diagnostics bug - numba intersect1d

[bednarsky]

Hi Stephan, here the bug:

file: logs/logs_slurm/plot_umap_diagnostics_method=densMAP,parameters=euclidean_15_0.1_2,sample=cancer__primary.err error:

Traceback (most recent call last): File "path/to/projects/project/modules/unsupervised_analysis/.snakemake/scripts/tmpgpjrocx8.plot_umap_diagnostics.py", line 41, in umap.plot.diagnostic(umap_obj, diagnostic_type='neighborhood', nhood_size=min(umap_obj.n_neighbors, 15), ax=ax_diag[1,1]) File "/path/to/snakemake_conda/4ba29b5deef3de008651353701702e01_/lib/python3.9/site-packages/umap/plot.py", line 1124, in diagnostic accuracy = nhood_compare( File "/path/to/snakemake_conda/4ba29b5deef3de008651353701702e01/lib/python3.9/site-packages/numba/core/dispatcher.py", line 468, in compile_for_args error_rewrite(e, 'typing') File "/path/to/snakemake_conda/4ba29b5deef3de008651353701702e01/lib/python3.9/site-packages/numba/core/dispatcher.py", line 409, in error_rewrite raise e.with_traceback(None) numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend) No implementation of function Function(<function intersect1d at 0x1555481dcca0>) found for signature:

intersect1d(array(int32, 1d, C), array(int32, 1d, C), assume_unique=Literalbool)

There are 2 candidate implementations:

• Of which 2 did not match due to: Overload in function 'jit_np_intersect1d': File: numba/np/arraymath.py: Line 3586. With argument(s): '(array(int32, 1d, C), array(int32, 1d, C), assume_unique=bool)': Rejected as the implementation raised a specific error: TypingError: got an unexpected keyword argument 'assume_unique' raised from /path/to/snakemake_conda/4ba29b5deef3de008651353701702e01_/lib/python3.9/site-packages/numba/core/typing/templates.py:784

During: resolving callee type: Function(<function intersect1d at 0x1555481dcca0>) During: typing of call at /path/to/snakemake_conda/4ba29b5deef3de008651353701702e01_/lib/python3.9/site-packages/umap/plot.py (209)

File "../../../../../../path/to/snakemake_conda/4ba29b5deef3de008651353701702e01_/lib/python3.9/site-packages/umap/plot.py", line 209: def _nhood_compare(indices_left, indices_right):

for i in range(indices_left.shape[0]): intersection_size = np.intersect1d(indices_left[i], indices_right[i], ^

My sample sheet looks like this

name,data,metadata,samples_by_features
subset_name,/path/to/results/demultiplexing/first_batch_of_samples/scvi/X_scVI__subset_name.csv,/path/to/results/demultiplexing/first_batch_of_samples/unsupervised_analysis/subset_name/labels.csv,1

My data file are the scVI coordinates

,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29
sID367_AAACAGCCAAGTTATC-1,0.8766483,0.011868089,-0.14010176,0.0021616258,-1.7530527,0.8266239,0.05426112,-0.29800522,-0.5499921,0.40404356,0.55395395,-0.011951547,0.070310116,0.1317476,0.09836078,-1.216123,-0.9669612,0.44787252,1.2134984,-1.6698662,-0.88864696,-0.31392187,-0.18586576,-1.0976224,-0.89937776,0.7491747,-0.39786023,-0.3978194,0.009200797,-0.009404337
sID367_AAACATGCAACTAGCC-1,1.2735096,-1.2301883,0.4899947,0.004316354,-0.4477949,-1.0801263,-0.41472286,-0.10565293,-1.0443822,0.1124156,0.71335185,0.01858944,0.069815695,1.8595614,0.9100859,-0.7941134,0.13103442,-0.38214165,0.01599136,0.719963,-1.0942267,0.033875763,0.5481672,-0.029896438,-1.0036578,-0.7464532,-0.04965532,0.33992022,0.016930878,0.016150381
sID367_AAACATGCACATGCTA-1,0.25002998,-0.112220734,-0.36979952,0.027052928,-0.23896998,-0.51691395,1.0869765,1.0108525,-0.81537515,0.71203756,-0.94883174,-0.014021037,0.07627165,-1.5595407,-0.6811844,-0.051620245,1.4360468,0.37079245,0.6642489,1.3201993,0.53024554,1.7682714,1.0888612,-0.40217578,-0.3562716,-0.63303614,0.22093366,0.09114313,0.008224259,-0.0056118146

The metadata file is a csv with multiple categorical columns but also numerical columns like gene_module_scores

Thank you for your help and the amazing pipelines!

[sreichl]

Thanks for reporting!

Did everything else run? No job depends on this rule, so it would be interesting if it is a reoccurring error or only happens in this specific rule.

[bednarsky]

It failed due to this error and shut down, no other errors shown (at least for the digits example data - with my data, when setting diagnostics: 0 other bugs occur - where at least one was due to a mistake I made and found I think)

[sreichl]

Okay thanks! Keep me posted. ;-)

[sreichl]

I think I have solved this now by fixing package compatibilities/dependencies.