IntracellularRecordingsTable stimulus/response require metadata even if the stimulus/response is missing

[oruebel]

This issue was originally reported by @lvsltz in https://github.com/oruebel/ndx-icephys-meta/issues/66. This issue has been copied here since it will need to be addressed in PyNWB.

The stimuli table (IntracellularStimuliTable) (and co.) requires stimulus_metadata to be passed even if the stimulus argument itself is None. I understand why this is happening, but it doesn't seem like an expected behavior.

Example from README identical until line (A):

ir_table = nwbfile.get_intracellular_recordings()
ir_table.add_column('new_column', 'Stimulus special column', category='stimuli')
meta = {}
meta['new_column'] = 2
ir_index = nwbfile.add_intracellular_recording(electrode=electrode,
                                               stimulus=stimulus,
                                               response=response,
                                               stimulus_metadata = meta)

# All going fine until here
ir_index = nwbfile.add_intracellular_recording(electrode=electrode,
                                               response=response)

Error:

ValueError: row data keys don't match available columns
you supplied 0 extra keys: set()
and were missing 1 keys: {'new_column'}

Checklist

• [X] Have you ensured the feature or change was not already reported?
• [X] Have you included a brief and descriptive title?
• [X] Have you included a clear description of the problem you are trying to solve?
• [X] Have you included a minimal code snippet that reproduces the issue you are encountering?
• [X] Have you checked our Contributing document?

[oruebel]

Implementing this feature will require:

• When setting data one will need to automatically determine appropriate "None" values for all columns of the corresponding table to fill in the row. Depending on the type of column this may be tricky to do automatically in all cases. For common types, e.g., int, str, float etc. this can be done, but for more complex types this may not be possible to do automatically. In this case we may need to ask the user to specify what None means.
• On read we should use numpy MaskedArray and MaskedConstant to mask the values of rows that do not specify a stimulus/response (i.e., for which the idx_start and count are -1). This should be a configurable option though, in case a user specifies actually valid metadata columns even in the case of a missing stimulus/response.