spyking-circus
spyking-circus copied to clipboard
Published
20 hours ago •
spyking-circus
spyking-circus
Spike sort in Docker containers with spiking-circus
Hello, I am trying to run the docker andI have an error when running the line :
sorting_SC = ss.run_spykingcircus(recording_loaded, output_folder="results_SC",
docker_image="spikeinterface/spyking-circus-base:1.0.7",
verbose=True)
I installed docker and had docker python SDK installed too. The error I had is the following:
Starting container
Installing spikeinterface==0.93.0 in spikeinterface/spyking-circus-base:1.0.7
Running spykingcircus sorter inside spikeinterface/spyking-circus-base:1.0.7
Stopping container
/Users/ismail/opt/anaconda3/envs/si090/lib/python3.8/pathlib.py:693: ResourceWarning: unclosed <socket.socket fd=131, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/ismail/.docker/run/docker.sock>
self = object.__new__(cls)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/ismail/opt/anaconda3/envs/si090/lib/python3.8/pathlib.py:693: ResourceWarning: unclosed <socket.socket fd=133, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/ismail/.docker/run/docker.sock>
self = object.__new__(cls)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/ismail/opt/anaconda3/envs/si090/lib/python3.8/pathlib.py:693: ResourceWarning: unclosed <socket.socket fd=135, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/ismail/.docker/run/docker.sock>
self = object.__new__(cls)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/ismail/opt/anaconda3/envs/si090/lib/python3.8/pathlib.py:693: ResourceWarning: unclosed <socket.socket fd=137, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/ismail/.docker/run/docker.sock>
self = object.__new__(cls)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/ismail/opt/anaconda3/envs/si090/lib/python3.8/pathlib.py:693: ResourceWarning: unclosed <socket.socket fd=139, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0, raddr=/Users/ismail/.docker/run/docker.sock>
self = object.__new__(cls)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
---------------------------------------------------------------------------
Exception Traceback (most recent call last)
Input In [93], in <cell line: 1>()
----> 1 sorting_SC = ss.run_spykingcircus(recording_loaded, output_folder="results_SC",
2 docker_image="spikeinterface/spyking-circus-base:1.0.7",
3 verbose=True)
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/sorters/runsorter.py:461, in run_spykingcircus(*args, **kwargs)
460 def run_spykingcircus(*args, **kwargs):
--> 461 return run_sorter('spykingcircus', *args, **kwargs)
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/sorters/runsorter.py:56, in run_sorter(sorter_name, recording, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, docker_image, singularity_image, with_output, **sorter_params)
49 """
50 Generic function to run a sorter via function approach.
51
52 >>> sorting = run_sorter('tridesclous', recording)
53 """ + _common_param_doc
55 if docker_image is not None:
---> 56 return run_sorter_container(sorter_name, recording, 'docker', docker_image,
57 output_folder=output_folder,
58 remove_existing_folder=remove_existing_folder,
59 delete_output_folder=delete_output_folder, verbose=verbose,
60 raise_error=raise_error, with_output=with_output, **sorter_params)
61 if singularity_image is not None:
62 return run_sorter_container(sorter_name, recording, 'singularity', singularity_image,
63 output_folder=output_folder,
64 remove_existing_folder=remove_existing_folder,
65 delete_output_folder=delete_output_folder, verbose=verbose,
66 raise_error=raise_error, with_output=with_output, **sorter_params)
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/sorters/runsorter.py:355, in run_sorter_container(sorter_name, recording, mode, container_image, output_folder, remove_existing_folder, delete_output_folder, verbose, raise_error, with_output, **sorter_params)
353 else:
354 if with_output:
--> 355 sorting = SorterClass.get_result_from_folder(output_folder)
357 if delete_output_folder:
358 shutil.rmtree(output_folder)
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/sorters/basesorter.py:256, in BaseSorter.get_result_from_folder(cls, output_folder)
252 if bool(log['error']):
253 raise SpikeSortingError(
254 "Spike sorting failed. You can inspect the runtime trace in spikeinterface_log.json")
--> 256 sorting = cls._get_result_from_folder(output_folder)
258 recording = load_extractor(output_folder / 'spikeinterface_recording.json')
259 if recording is not None:
260 # can be None when not dumpable
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/sorters/spyking_circus/spyking_circus.py:169, in SpykingcircusSorter._get_result_from_folder(cls, output_folder)
167 @classmethod
168 def _get_result_from_folder(cls, output_folder):
--> 169 sorting = SpykingCircusSortingExtractor(folder_path=Path(output_folder) / 'recording')
170 return sorting
File ~/opt/anaconda3/envs/si090/lib/python3.8/site-packages/spikeinterface/extractors/spykingcircusextractors.py:53, in SpykingCircusSortingExtractor.__init__(self, folder_path)
51 break
52 if results is None:
---> 53 raise Exception(spykingcircus_folder, " is not a spyking circus folder")
55 # load params
56 sample_rate = None
Exception: (PosixPath('/Users/ismail/Desktop/PDS/Official_Tutorial_SI_0.90_Aug2021/results_SC/recording'), ' is not a spyking circus folder')
The content of the log folder is :
--------------------- Informations -------------------------
| Number of recorded channels : 32
| Number of analyzed channels : 32
| File format : NUMPY
| Data type : float32
| Sampling rate : 30000 Hz
| Duration of the recording : 2 min 27 s 618 ms
| Width of the templates : 3 ms
| Spatial radius considered : 100 um
| Threshold crossing : negative
------------------------------------------------------------------
------------------------- Informations -------------------------
| Using only 2 out of 4 local CPUs available (-c to change)
------------------------------------------------------------------
[37mFiltering with a Butterworth filter (order 3) in [300, 14250] Hz
0%| |[00:00<?, ?it/s]
33%|████████████ |[00:48<01:36, 48.08s/it]
67%|████████████████████████ |[01:37<00:48, 48.65s/it]
100%|████████████████████████████████████|[02:09<00:00, 41.12s/it]
100%|████████████████████████████████████|[02:09<00:00, 43.10s/it]
[37mAnalyzing data to get whitening matrices and thresholds...
[37mFound 30s to compute the whitening matrix...
[37mBecause of whitening, need to recompute the thresholds...
[37mSearching spikes to construct the PCA basis...
0%| |[00:00<?, ?it/s]
100%|████████████████████████████████████|[01:29<00:00, 89.72s/it]
100%|████████████████████████████████████|[01:29<00:00, 89.72s/it]
[37mFound 11648 waveforms over 25600 requested
[33m------------------------- Informations -------------------------
[33m| A basis with 5 dimensions has been built
[33m------------------------------------------------------------------
[37m
[37mSearching isolated random spikes to sample amplitudes...
0%| |[00:00<?, ?it/s]
100%|███████████████████████████████████|[02:09<00:00, 129.20s/it]
100%|███████████████████████████████████|[02:09<00:00, 129.20s/it]
[37mFound 14075 spikes over 256000 requested
[37mEstimating amplitudes distributions...
[37mSmart Search of good isolated spikes for the clustering (1/3)...
0%| |[00:00<?, ?it/s]
100%|███████████████████████████████████|[02:16<00:00, 136.79s/it]
100%|███████████████████████████████████|[02:16<00:00, 136.79s/it]
[37mFound 14075 isolated spikes over 256000 requested (0 rejected)
[37mComputing density estimations...
[37mSearching random spikes to refine the clustering (2/3)...
0%| |[00:00<?, ?it/s]
100%|████████████████████████████████████|[00:48<00:00, 48.70s/it]
100%|████████████████████████████████████|[00:48<00:00, 48.71s/it]
[37mFound 17 spikes over 256000 requested
[37mRefining density estimations...
[37mSearching random spikes to refine the clustering (3/3)...
0%| |[00:00<?, ?it/s]
100%|████████████████████████████████████|[00:50<00:00, 50.20s/it]
100%|████████████████████████████████████|[00:50<00:00, 50.20s/it]
[37mFound 0 spikes over 256000 requested
[33m------------------------- Informations -------------------------
[33m| No more spikes in the recording, stop searching
[33m------------------------------------------------------------------
[37m
[37mRefining density estimations...
[37mRunning density-based clustering...
0%| |[00:00<?, ?it/s]
6%|██▎ |[00:00<00:03, 4.07it/s]
12%|████▌ |[00:00<00:02, 6.19it/s]
19%|██████▊ |[00:02<00:13, 1.02s/it]
25%|█████████ |[00:02<00:07, 1.50it/s]
31%|███████████▎ |[00:09<00:33, 3.08s/it]
38%|█████████████▌ |[00:10<00:20, 2.10s/it]
44%|███████████████▊ |[00:10<00:13, 1.45s/it]
50%|██████████████████ |[00:10<00:08, 1.03s/it]
56%|████████████████████▎ |[00:10<00:05, 1.30it/s]
69%|████████████████████████▊ |[00:10<00:02, 2.22it/s]
75%|███████████████████████████ |[00:10<00:01, 2.56it/s]
81%|█████████████████████████████▎ |[00:11<00:01, 1.83it/s]
88%|███████████████████████████████▌ |[00:12<00:01, 1.93it/s]
94%|█████████████████████████████████▊ |[00:12<00:00, 2.39it/s]
100%|████████████████████████████████████|[00:12<00:00, 1.27it/s]
[33m------------------------- Informations -------------------------
[33m| Number of clusters found : 29
[33m| Number of local merges : 0 (method nd-bhatta, param 2)
[33m| Not enough spikes gathered: -put safety_space=False?
[33m| -remove smart_search?
[33m------------------------------------------------------------------
[37m
[37mEstimating the templates with the median-raw procedure ...
0%| |[00:00<?, ?it/s]
6%|██▎ |[00:01<00:24, 1.64s/it]
19%|██████▊ |[00:05<00:23, 1.77s/it]
25%|█████████ |[00:06<00:20, 1.75s/it]
31%|███████████▎ |[00:08<00:19, 1.78s/it]
38%|█████████████▌ |[00:10<00:17, 1.75s/it]
56%|████████████████████▎ |[00:12<00:07, 1.07s/it]
69%|████████████████████████▊ |[00:13<00:05, 1.00s/it]
75%|███████████████████████████ |[00:18<00:06, 1.68s/it]
81%|█████████████████████████████▎ |[00:23<00:07, 2.49s/it]
88%|███████████████████████████████▌ |[00:25<00:04, 2.30s/it]
94%|█████████████████████████████████▊ |[00:27<00:02, 2.14s/it]
100%|████████████████████████████████████|[00:27<00:00, 1.69s/it]
[37mMerging similar templates...
0%| |[00:00<?, ?it/s]
6%|██▎ |[00:00<00:11, 1.32it/s]
19%|██████▊ |[00:01<00:05, 2.29it/s]
25%|█████████ |[00:02<00:05, 2.00it/s]
31%|███████████▎ |[00:02<00:06, 1.80it/s]
38%|█████████████▌ |[00:03<00:05, 1.73it/s]
56%|████████████████████▎ |[00:03<00:02, 2.76it/s]
69%|████████████████████████▊ |[00:04<00:01, 2.93it/s]
75%|███████████████████████████ |[00:05<00:01, 2.48it/s]
81%|█████████████████████████████▎ |[00:05<00:01, 2.18it/s]
88%|███████████████████████████████▌ |[00:06<00:00, 2.00it/s]
94%|█████████████████████████████████▊ |[00:07<00:00, 1.89it/s]
100%|████████████████████████████████████|[00:07<00:00, 2.26it/s]
[37mRemoving mixtures of templates...
0%| |[00:00<?, ?it/s]
6%|██▎ |[00:00<00:10, 1.49it/s]
19%|██████▊ |[00:01<00:05, 2.42it/s]
25%|█████████ |[00:01<00:05, 2.08it/s]
31%|███████████▎ |[00:02<00:05, 1.87it/s]
38%|█████████████▌ |[00:03<00:05, 1.79it/s]
56%|████████████████████▎ |[00:03<00:02, 2.83it/s]
69%|████████████████████████▊ |[00:04<00:01, 2.97it/s]
75%|███████████████████████████ |[00:05<00:01, 2.48it/s]
81%|█████████████████████████████▎ |[00:05<00:01, 2.12it/s]
88%|███████████████████████████████▌ |[00:06<00:01, 1.93it/s]
94%|█████████████████████████████████▊ |[00:07<00:00, 1.84it/s]
100%|████████████████████████████████████|[00:07<00:00, 2.27it/s]
0%| |[00:00<?, ?it/s]
23%|████████▎ |[00:00<00:00, 24.23it/s]
46%|████████████████▌ |[00:00<00:00, 25.38it/s]
69%|████████████████████████▉ |[00:00<00:00, 24.84it/s]
92%|█████████████████████████████████▏ |[00:00<00:00, 25.16it/s]
100%|████████████████████████████████████|[00:00<00:00, 25.06it/s]
[33m------------------------- Informations -------------------------
[33m| Number of global merges : 3
[33m| Number of mixtures removed : 1
[33m------------------------------------------------------------------
[37m
[37mComputing optimal amplitudes for the templates...
0%| |[00:00<?, ?it/s]
8%|██▊ |[00:08<01:45, 8.77s/it]
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 139 RUNNING AT 94486e4b2ebb
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
---------------------------- Error -----------------------------
| Step "clustering" failed for reason Command '['mpiexec', '-np', '2', 'spyking-circus-subtask', 'clustering', '/Users/ismail/Desktop/PDS/Official_Tutorial_SI_0.90_Aug2021/results_SC/recording.npy', '2', '0', 'False', 'False']' returned non-zero exit status 9.!
Thank you very much for your help.
Strange. Looks like the code is crashing, but this is rather small and not that challenging dataset, at a first glimpse. Could you send it to me?
Of course ! You can find the recording I used here : https://drive.google.com/file/d/1PzghSGv9EgcPU2uczahPXh2sozb8psgb/view?usp=sharing
I posted a relatively similar issue in the spike-interface page : https://github.com/SpikeInterface/spikeinterface/issues/1034
