error trying to load waveforms from existing folder

[Yizhar-Lab]

Hi, Running

we = si.extract_waveforms(recording, sorting, wf_dir, load_if_exists=True, overwrite=False, 
                              ms_before=0.25, ms_after=0.75, max_spikes_per_unit=1e5, n_jobs=1, chunk_size=30000)

raises ValueError: spikeinterface.Base.load() file_path must be an existing folder or file, though I just created this folder using the same call exept for load_if_exists=False, overwrite=True. Thanks!

[alejoe91]

Can you paste the entire traceback? You have your recording and sorting.json in the folder? What type of recording are you using?

[Yizhar-Lab]

Thanks for the fast response @alejoe91! It seems that extract_waveforms creates recording.json and params.json, but not sorting.json in this folder, which corresponds to a sorting I created using se.NumpySorting.from_times_labels() based on a manual sorting I did previously. Contrastly, when I extract_waveforms from a sorting created by ss.run_kilosort2(), I do get a sorting.json in the waveforms folder. The recording is created using BinaryRecordingExtractor(rec_file, fs, num_chans, 'int16') where rec_file refers to a standard flat binary file with 16 channels and 31.25 kHz.

Traceback (most recent call last):

  File "e:\auto_spike_sorting\code\comparemanaulautosorting.py", line 154, in <module>
    we_asaf, pc_asaf, metrics_asaf = get_quality_metrics(recording, asaf_sort, wf_dir_asaf, pc_dir_asaf,

  File "e:\auto_spike_sorting\code\comparemanaulautosorting.py", line 77, in get_quality_metrics
    we = si.extract_waveforms(recording, sorting, wf_dir+'\\waveforms',

  File "c:\users\yizharanalysis\anaconda3\envs\spike_sorting\lib\site-packages\spikeinterface\core\waveform_extractor.py", line 685, in extract_waveforms
    we = WaveformExtractor.load_from_folder(folder)

  File "c:\users\yizharanalysis\anaconda3\envs\spike_sorting\lib\site-packages\spikeinterface\core\waveform_extractor.py", line 92, in load_from_folder
    recording = load_extractor(folder / 'recording.json')

  File "c:\users\yizharanalysis\anaconda3\envs\spike_sorting\lib\site-packages\spikeinterface\core\base.py", line 757, in load_extractor
    return BaseExtractor.load(file_or_folder_or_dict, base_folder=base_folder)

  File "c:\users\yizharanalysis\anaconda3\envs\spike_sorting\lib\site-packages\spikeinterface\core\base.py", line 493, in load
    raise ValueError('spikeinterface.Base.load() file_path must be an existing folder or file')

ValueError: spikeinterface.Base.load() file_path must be an existing folder or file

[alejoe91]

Ahhh! That's the problem! The NumpySorting is not dumpable, as it has all spike times and labels in memory. To solve this you can first save the sorting to to a folder - internally it saves the spiking info to npz (sorting = sorting.save(folder='my-folder')) and then use the returned sorting object, which is now dumpable as it's been saved to a file. I hope this clears things up :)

[alejoe91]

Anyways, we should have better checks and more informative error messages on our side, so thanks for opening the issue!