AB: For the case all sweeps are loaded from an experiment add fast code path
In the AB it can happen that a user wants to load all sweeps from an experiment.
Currently index over every single sweep selected and load sequentially. This could be sped up for the case that all sweeps are loaded by loading the data folder with the sweep data recursively in a single call.
Is that an issue for PXP and NWB?
Edit: Yes it is. And AB_LoadSweepFromNWB should also not load the stimulus.
The stimulus comment in AB_LoadSweepFromNWB refers to the Sweep DA channel data waves. It does not load the stimset.
Loading algorithm:
The current algorithm is rather dumb and iterates over all marked rows of the list and attempts to load for each entry in AB_LoadFromFile. If it is an expandable row then AB_LoadFromExpandedRange is called, otherwise the single entity is loaded. The code for the load calls is basically duplicated in AB_LoadFromExpandedRange and AB_LoadFromFile. It also has the effect that if an expandable row and a single row include the same sweep, then it is attempted to be loaded twice.
Reasonable changes: Split the loading in three phases:
- Gather all entities to be loaded from the rows, e.g. analysisbrowser mapindizes with loadtype.
- Remove duplicates and separate on a per file basis
- Load data
This allows to add multiple optimizations:
- Because the prepared loader information is arranged by file, we can open NWB files once and reuse the h5FileId in step 3. Currently we open and close the file for each entity loaded.
- In step 2 we can determine special cases where e.g. all sweeps from a file should be loaded and handle that more efficiently in step 3.