spikeinterface
spikeinterface copied to clipboard
SpikeInterface
Extra neo kwarg for BioCAM's event-based compression not passed to SpykingCircus2
Recently, support for BioCAM's event-based compression was added to neo.
It requires the user to decide between filling the gaps in recordings with zeros or artificial noise, specified by fill_gaps_strategy in SI.
When running SpykingCircus2 on compressed data, it looks like this kwarg isn't passed properly to the internal filtering functions.
I'm running the following :
recording = se.read_biocam(data_path,
mea_pitch=60,
electrode_width=21,
fill_gaps_strategy="synthetic_noise",
)
recording = recording.remove_channels('1')
recording = spre.unsigned_to_signed(recording, bit_depth=12)
sorting = ss.run_sorter(
sorter_name="spykingcircus2", recording=recording,
folder=str(output_folder / "sc2_output"),verbose=True, **p,
)
HerdingSpikes2 runs just fine on the same file, so this seems to be sorter-specific.
Here's the full trace :
Error running spykingcircus2
Traceback (most recent call last):
File "/home/user/Documents/ephy/sc_si.py", line 60, in <module>
sorting = ss.run_sorter(
^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 198, in run_sorter
return run_sorter_local(**common_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/sorters/runsorter.py", line 260, in run_sorter_local
SorterClass.run_from_folder(folder, raise_error, verbose)
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/sorters/basesorter.py", line 310, in run_from_folder
raise SpikeSortingError(
spikeinterface.sorters.utils.misc.SpikeSortingError: Spike sorting error trace:
Traceback (most recent call last):
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/sorters/basesorter.py", line 270, in run_from_folder
SorterClass._run_from_folder(sorter_output_folder, sorter_params, verbose)
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/sorters/internal/spyking_circus2.py", line 172, in _run_from_folder
recording_w = whiten(recording_f, **whitening_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/core/core_tools.py", line 35, in source_class_or_dict_of_sources_classes
return source_class(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/whiten.py", line 93, in __init__
W, M = compute_whitening_matrix(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/whiten.py", line 192, in compute_whitening_matrix
data, cov, M = compute_covariance_matrix(recording, apply_mean, regularize, regularize_kwargs, random_chunk_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/whiten.py", line 244, in compute_covariance_matrix
random_data = get_random_data_chunks(recording, concatenated=True, return_in_uV=False, **random_chunk_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/core/recording_tools.py", line 608, in get_random_data_chunks
traces_chunk = recording.get_traces(
^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/core/baserecording.py", line 339, in get_traces
traces = rs.get_traces(start_frame=start_frame, end_frame=end_frame, channel_indices=channel_indices)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/common_reference.py", line 183, in get_traces
traces = self.parent_recording_segment.get_traces(start_frame, end_frame, slice(None))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/filter.py", line 162, in get_traces
traces_chunk, left_margin, right_margin = get_chunk_with_margin(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/core/recording_tools.py", line 854, in get_chunk_with_margin
traces_chunk = rec_segment.get_traces(
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/preprocessing/unsigned_to_signed.py", line 54, in get_traces
traces = self.parent_recording_segment.get_traces(start_frame, end_frame, channel_indices)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/core/channelslice.py", line 101, in get_traces
traces = self._parent_recording_segment.get_traces(start_frame, end_frame, parent_indices)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/spikeinterface/src/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 367, in get_traces
raw_traces = self.neo_reader.get_analogsignal_chunk(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/python-neo/neo/rawio/baserawio.py", line 1006, in get_analogsignal_chunk
raw_chunk = self._get_analogsignal_chunk(block_index, seg_index, i_start, i_stop, stream_index, channel_indexes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/Documents/GitHub/python-neo/neo/rawio/biocamrawio.py", line 148, in _get_analogsignal_chunk
raise ValueError(
ValueError: Please set `fill_gaps_strategy` to 'zeros' or 'synthetic_noise'.
I think that it might be missing from the self._kwargs...can you check @b-grimaud ?
I looked into it a bit more, the recording goes through the first few filtering functions just fine until whitening. I'll try to see if any kwargs are passed differently compared to CMR or bandpass filtering.
@b-grimaud,
Alessio is referring to this:
https://github.com/SpikeInterface/spikeinterface/blob/fb72ed2af963e257a2a12b1d4280f622c259c7c3/src/spikeinterface/extractors/neoextractors/biocam.py#L48-L67
We need to pass the new kwargs from spikeinterface into neo during the __init__ of the extractor. No one has updated the spikeinterface extractor after your neo PR :) Want to fix that and see if that helps first?
@zm711 I think you were correct still, I dug into it and that neo kwarg does not get dumped in the spikeinterface_recording JSON.
When the recording is re-loaded from folder by the sorter, the kwarg is lost.
What I'm not sure about is why it's not happening or how it's supposed to work. I looked into other neo extractors for reference but I'm not familiar with them. For now I've just added fill_gaps_strategy to the self._kwargs.update section and it works, but this might not be a proper fix.