spikeinterface
spikeinterface copied to clipboard
SpikeInterface
Updated WaveformExtractor.load_from_folder
Added the option to give a recording and/or sorting object as to not reload them if they already exist (gaining quite a bit of time and memory).
@Dradeliomecus
We have been working on this PR #990 in order to be able to skip the loading of the recording completely. We expose most of the metadata asociated to the recording directly as we properties. However, I think that loading the sorting from the json file is lightning fast.
Do you have some performance metrics?
Memory-wise, I agree this could be an overhead. But the load_from_folder function assumes that you are reloading the objects, so I think the behavior is correct.
For example in my case I just want to load the templates (and nothing else), which have already been computed and saved in the folder.
For another thing I'm doing, I have 252 PhySortingExtractor, which take ~7 minutes to load in total (and take around 25 GB of memory). They all correspond to the same recording though. In this application I'm not using the WaveformExtractor, because reloading it from folder for each analysis would take too much time and memory (I save the templates file myself).
For example in my case I just want to load the templates (and nothing else), which have already been computed and saved in the folder.
For another thing I'm doing, I have 252
PhySortingExtractor, which take ~7 minutes to load in total (and take around 25 GB of memory). They all correspond to the same recording though. In this application I'm not using the WaveformExtractor, because reloading it from folder for each analysis would take too much time and memory (I save the templates file myself).
Fair enough! then I'm ok with it!