[ENH] Add audio/video recordings to behavioral experiments

[bendichter]

fix #1771

edit by @yarikoptic: preview - https://bids-specification--2231.org.readthedocs.build/en/2231/modality-specific-files/behavioral-experiments.html

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests. :white_check_mark: Project coverage is 82.86%. Comparing base (7c06228) to head (aead0a4).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2231   +/-   ##
=======================================
  Coverage   82.86%   82.86%           
=======================================
  Files          20       20           
  Lines        1669     1669           
=======================================
  Hits         1383     1383           
  Misses        286      286           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[satra]

a colleague asked why all recording sources are being put into a single folder when other parts of bids are separated by modality/technique. perhaps a sentence or two for this rationale would be helpful to include in the specification.

[bendichter]

@satra The rationale here is that the organizing principle for the beh/ directory is the type of data being captured (behavior) rather than the recording technique (audio, video, events, etc.).

In the context of behavioral experiments, audio recordings (capturing vocalizations, speech), video recordings (capturing facial expressions, body movements), and event files all serve the same fundamental purpose: documenting subject behavior. They are different modalities of the same underlying phenomenon.

This is analogous to how func/ contains both BOLD and ASL data - different acquisition techniques measuring the same underlying phenomenon (functional brain activity).

Additionally, keeping these files together in beh/ makes it easier to:

1. Find all behavioral data for a session in one place
2. Maintain temporal alignment metadata between synchronized recordings
3. Keep behavioral data organized when collected alongside neural recordings (where the neural data lives in its respective modality folder like ieeg/ or eeg/)

I can add a sentence or two to the specification explaining this rationale. Would something like the following work?

"Audio and video recordings are stored together in the beh/ directory because they represent different modalities for capturing the same underlying phenomenon: subject behavior. This organization mirrors other BIDS directories where the organizing principle is the type of data being measured rather than the specific recording technique."

@satra did I understand the question? Do you think this would satisfy the colleague in the context of your conversation with them?

[bendichter]

See https://github.com/bids-standard/bids-examples/pull/523 for an example dataset that validates under this schema

[yarikoptic]

@neuromechanist this PR has lots of overlap with your work on

• https://github.com/bids-standard/bids-specification/pull/2022

Please review here and also the

• https://github.com/bids-standard/bids-examples/pull/523

since we should aim for consistency regarding the metadata on audio/video files etc

[neuromechanist]

My first suggestion (🤓) would be to request a BEP number for this PR from maintainers, as this is quite a change, and arguably/conceptually adds a new modality (although audio and video are being added in BEP044), which would require a BEP. I'll take it up to this week's maintainer meeting.

[bendichter]

@neuromechanist

OK, I have submitted a PR to the website to make this an official BEP here: https://github.com/bids-standard/bids-website/pull/759