SpikeInterface
Patch to skip chunks without peaks or if enough peaks have been detected
Could such a solution work @samuelgarcia ? This is a patch to try to address #2001. The idea is that if we have a PeakSource node, and no peaks between start_frame and end_frame, then we should clearly reduce IO by not loading the traces
Your patch is worst than before, you are loading (and computing) traces for all nodes!! We need the trick before the loop.
Ok, so what should be done is simply checking if the first node is a PeakSource, and it it has spikes, then load traces otherwise skip. I'll give it a try
Yes. We check the first node and we skip the entire grpah if threre is no peak. I propose t return None in that case so it is faster.
I've extended the PR to cases where we want to stop everything when at least max_peaks have been discovered. This is working, and to allow shuffliung of the chunks, we need to add some seed arguments to the run() function of the ChunkRecordingExecutor. By doing so, chunks can be shuffled, and as soon as max_peaks have been detected, we stop loading traces (or if no peaks are in the chunks)
Yes, @samuelgarcia wants to redesign something, so I guess this will be postponed