pynwb icon indicating copy to clipboard operation
pynwb copied to clipboard

Add conversion factor to waveform columns

Open CodyCBakerPhD opened this issue 2 years ago • 1 comments

Motivation

For large amounts of recording data, even adding waveform snippets to the columns of a Units table can be a significant task. One step towards reducing unnecessary data inflation is to add attributes for scaling factors similar to the behavior of other TimeSeries-like objects that allow the data to be stored in some minimal base type, while the conversion factor then scales it into the specified scientific units.

I've added these attributes and attempted to propagate them through the IO, mirroring the patterns established by the waveform_rate attribute.

Sister PR: https://github.com/NeurodataWithoutBorders/nwb-schema/pull/491

How to test the behavior?

Behavior is showcased in the steps, identical to waveform_rate. What is currently untested in both cases is the default behavior of merely calling nwbfile.add_unit() the first time in a fresh NWBFile, which auto-generates a blank Units table. We should discuss how attributes of that table ought to be set in that situation (probably just be always being sure to define nwbfile.Units = Units(**my_attributes) before adding any actual units).

Checklist

  • [ ] Did you update CHANGELOG.md with your changes?
  • [X] Have you checked our Contributing document?
  • [X] Have you ensured the PR clearly describes the problem and the solution?
  • [X] Is your contribution compliant with our coding style? This can be checked running flake8 from the source directory.
  • [X] Have you checked to ensure that there aren't other open Pull Requests for the same change?

CodyCBakerPhD avatar Jan 10 '22 18:01 CodyCBakerPhD

@rly This would be a bit more proper to utilize the MeasurementData extension to the VectorData being proposed at the higher hdmf level, and while it's clear how to specify that in the nwb-schema it's not as clear to me how to actually get the adjust the columns of the DynamicTable here in pynwb to use that.

CodyCBakerPhD avatar Jan 10 '22 18:01 CodyCBakerPhD