spikeinterface icon indicating copy to clipboard operation
spikeinterface copied to clipboard

warning messages during analyzer computations

Open shahafweissMPI opened this issue 2 months ago • 2 comments

Hello Devs, i'm getting all kinds of scipy warnings during the analyzer calculations, and was wondering if this is normal, or do i have an issue with scipy / spike data format?

estimate_sparsity engine=thread - n_jobs=6 - samples_per_chunk=30,000 - chunk_memory=21.97 MiB - total_memory=131.84 MiB - chunk_duration=1.00s (999.99 ms) estimate_sparsity (workers: 6 threads): 0%| | 0/6999 [00:00<?, ?it/s] 2025-11-07 09:38:39.506 | WARNING | spikeinterface.core.base:save_to_folder:967 - G:\mambaroot\envs\hir_new\Lib\site-packages\spikeinterface\core\base.py:967: UserWarning: The extractor is not serializable to file. The provenance will not be saved. warnings.warn("The extractor is not serializable to file. The provenance will not be saved.")

2025-11-07 09:38:40.777 | WARNING | spikeinterface.core.sortinganalyzer:create_binary_folder:497 - G:\mambaroot\envs\hir_new\Lib\site-packages\spikeinterface\core\sortinganalyzer.py:497: UserWarning: The sorting provenance is not serializable! The sorting provenance link will be lost for future load warnings.warn(

new analyzer created at //gpfs.corp.brain.mpg.de/stem/data/project_hierarchy/data/afm16924/240525/trial0/postscript_2025KS/sorting_analyzer_results noise_level (workers: 6 threads): 0%| | 0/20 [00:00<?, ?it/s] compute_waveforms (workers: 6 threads): 0%| | 0/6999 [00:00<?, ?it/s] computing template simliarity and metrics 2025-11-07 10:00:58.659 | WARNING | scipy.stats._stats_py:linregress:10730 - C:\Users\su-weisss\AppData\Roaming\Python\Python311\site-packages\scipy\stats_stats_py.py:10730: RuntimeWarning: invalid value encountered in scalar divide slope = ssxym / ssxm

2025-11-07 10:00:58.660 | WARNING | scipy.stats._stats_py:linregress:10744 - C:\Users\su-weisss\AppData\Roaming\Python\Python311\site-packages\scipy\stats_stats_py.py:10744: RuntimeWarning: invalid value encountered in sqrt t = r * np.sqrt(df / ((1.0 - r + TINY)*(1.0 + r + TINY)))

2025-11-07 10:00:58.661 | WARNING | scipy.stats._stats_py:linregress:10750 - C:\Users\su-weisss\AppData\Roaming\Python\Python311\site-packages\scipy\stats_stats_py.py:10750: RuntimeWarning: invalid value encountered in scalar divide slope_stderr = np.sqrt((1 - r**2) * ssym / ssxm / df)

shahafweissMPI avatar Nov 07 '25 09:11 shahafweissMPI

Hello, yeah, these are common warnings when computing template metrics. It will be something like: one of the metrics computes a slope of a template, which involves a division, and when it's doing this for a junk unit there's a division by zero. Or one of the metrics makes an assumption about the shape of a template, this assumption fails for a certain unit, and causes a division by zero. You're getting an error for each unit that fails, so if you're computing 40 metrics for 600 units, there are tens of thousands of opportunities for warnings! It's not a problem - you'll end up with a bunch of nans for some units for some metrics.

We could do better as devs by catching these errors in the template metric algorithms before they happen.

chrishalcrow avatar Nov 07 '25 10:11 chrishalcrow

Maybe the new refactoring of @alejoe91 will help a bit.

samuelgarcia avatar Nov 14 '25 14:11 samuelgarcia