How to save waveforms from spikeinterface to NWB?

[khl02007]

Is there a function to save waveforms in spikeinterface (SortingAnalyzer extension or older WaveformExtractor) to NWB format?

[alejoe91]

Hi @khl02007!

You can use the write_sorting_analyzer from the neuroconv package!

from neuroconv.tools.spikeinterface.spikeinterface import write_sorting_analyzer

write_sorting_analyzer(sorting_analyzer, nwbfile_path="path-to-nwb-file")

You can also write to an open nwbfile using the nwbfile argument or simply add the sorting analyzer info (units table + waveform means/stds + quality/template metrics column) to an open nwbfile without writing it with the add_sorting_analyzer_to_nwbfile function.

[h-mayorquin]

Clarification for people that were confused like me: Note that this adds the means and the stds of the waveforms for each units as @alejoe91 mentions above. It does not add all the waveforms which is probably a good idea since that will make the NWB explode in size and anyway you can quickly extract each of the individual waveforms on the fly as you have the timings on the units table.

[borrepp]

Hi Heberto (@h-mayorquin )

One question, which might be more related to NWB than spikeinterface.

Once that the UNITS table is created (without the waveforms column). How can we retrieve the individual waveforms? The NWB.electricalseries are likely stored in the acquisition (i.e. raw recording) or ephys.FilteredEphys (i.e. spikeinterface preprocessed recording) containers. I searched for some example code/function but didn't find any.

Thanks Pepe

[alejoe91]

Hi @borrepp

Individual waveforms are not saved. Only the mean and std of the waveforms are added as columns.

[borrepp]

Hi @alejoe91 Thanks, I understood that part. My question is more on the NWB side. Once that I exported the sorting_analyzer (recording & units) to an NWB file. How can I read the waveforms? Or, for example, is there a function to create a sorting_analyzer from an NWB containing recording & units that will allow access to the waveforms? Should I post this question on the NWB or neuroconv site? Thanks in advance for any help. Best Pepe

[khl02007]

@alejoe91 @h-mayorquin Thanks so much!

I think it's worth having an option to save all the waveforms and not just the mean. For example, many quality metrics require all the waveforms (e.g. to project onto PC space). It would be too time-consuming to re-extract waveforms every time you want to compute a different metric. The sorting analyzer saves waveforms to disk anyway; maybe there could be an option to save it in NWB format?

[alejoe91]

Probably worth porting this issue over to neuroconv: https://github.com/catalystneuro/neuroconv

The "problem" is that by default the waveforms are only extracted on a subset of spikes, and I think that NWB would require waveforms for all spikes.

[h-mayorquin]

Once that I exported the sorting_analyzer (recording & units) to an NWB file. How can I read the waveforms? Or, for example, is there a function to create a sorting_analyzer from an NWB containing recording & units that will allow access to the waveforms?

Load the NWBRecording and the NWBSorting and create a sorting analyzer then you can use the extract waveforms functionality of the analyzer.

[zm711]

Has this reached its conclusion here? Or should we leave this issue open?

[borrepp]

From our side, I think so. Individual waveforms are not exported from sorting_anaylzer. Neither NWB has a function to do it.

I made some custom code to extract & write waveforms from each unit's timestamps after exporting into an NWB, but we stopped doing it because it was too slow. We will pull waveforms from NWB only when we really need them, which is rare.

Thanks for the follow-up @zm711

Best Pepe