how to store non-TwoPhotonSeries ophys?

[tbenst]

I'm not certain how to store light-sheet / light field / DIFF microscopy or even widefield, etc data in NWB. TwoPhotonSeries is perfectly reasonable, and uses an imaging_plane. ImageSeries does not support imaging_plane. Perhaps there should be a new, more generic OphysSeries?

For now, seems like using TwoPhotonSeries is my best bet?

[bendichter]

@tbenst Yes, our ophys module is still a little outdated, and should support sources other than TwoPhotonSeries. We have this open PR for OnePhotonSeries: https://github.com/NeurodataWithoutBorders/nwb-schema/issues/283. Are there other types of data types you would need? What attributes would they need?

[tbenst]

I'm not sure if 1P / 2P is right decision criteria. Multiphoton can be scanning (e.g. 2P microscope) with a single PMT, holographic illumination with CMOS sensor, light sheet with CMOS sensor, etc.

Instead of distinguishing by illumination wavelength, IMHO it should be based on the light sensor. The three most common are probably PMT, rolling shutter (ie CMOS), and global shutter (ie CMOS). The last two can probably be combined.

Additionally, something about the illumination might be included. A PMT series can mean something very different when scanned with a Bessel beam, for instance. Likewise, for voltage imaging using holograhic illumination, the imaging plane may stay constant, and recording done with same CMOS, but the actual image will change dramatically depending on which cells are illuminated. Thus, the depth being recorded from may change on a per-pixel basis, based on illumination.

Edit: for my purposes, just need imaging_plane for now--perhaps worth adding this to ImageSeries?

[bendichter]

Just so I am clear, are you suggesting that we simply mark PMT vs. roller shutter vs. global shutter, or would these different light sensors each have different meta-data requirements?

[tbenst]

Yes, I think you could put all three into the same FunctionalImageSeries or similar name. For my current usecases, that would work fine.

For others, they may have different metadata requirements in some cases: for example some 2P microscopes support random access using PMTs in a way that's not possible with CMOS sensors, so while much 2P data can reasonably be stored as (Z x) H x W, random access might be stored as just fluorescence of dim T with an accompanying array of the spatial index T x 3. I don't do this style of imaging so best ask someone else for preferred format, I just bring it up as an example how different image sensors have different constraints on how the data needs to be stored.

Edit: suppose that a sparse matrix may work as well, but would lose the intra-frame temporal information.

[bahanonu]

@bendichter @tbenst OphysSeries seems like the most appropriate name. Would anticipate the majority of imaging situations and/or users will use that general format ([frame x y] or [frame x y z]) while specialized cases (e.g. random access) can be planned out or designed around that generic class.