MIES icon indicating copy to clipboard operation
MIES copied to clipboard

Add epoch recreation export in NWB2 for mass processing

Open MichaelHuth opened this issue 1 year ago • 2 comments

from https://github.com/AllenInstitute/MIES/pull/2041#issuecomment-2017914543

I've chatted with Tim about this and here is our conclusion:

Overwrite the existing epoch info in NWBv2 files, existing IPNWB functions: WriteSingleChannel, WriteEpochs, AppendEpochTable Should be done only on request (i.e. just implement a function) We will then do the epoch overwriting with an approach like MIES_MassExperimentProcessing.ipf

We can do that in another PR given that this PR is self-contained with tests.

Conceptionally it has to be solved that NWB2 export is threadsafe and epoch recreation is not. An idea would be to iterate through all sweeps before and cache/buffer the recreated epochs such that NWB2 export can access the data threadsafe.

MichaelHuth avatar Mar 27 '24 17:03 MichaelHuth

We always want to write recreated epoch info into the labnotebook if we don't have epoch info at all. We also want to extend the labnotebook with the new epoch info and store that into NWB. See #881.

t-b avatar Feb 03 '25 13:02 t-b

Extend NWB_ExportAllData

Insert (row) epoch info in labnotebook at the end of last sweep+sourceType block. (only this is relevant if there was a sweep rollback)

FindRange

Also add new column LBN to flag entry as "PostprocessedEntry"

MichaelHuth avatar Mar 31 '25 15:03 MichaelHuth