[ENH] microelectrode electrophysiology specification (BEP032)

[yarikoptic]

Replaces https://github.com/bids-standard/bids-specification/pull/1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal. old google doc

[!Note]

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure

[!Tip]

HTML preview of this BEP

---

• To use this WiP schema on sample datasets, see https://deno.land/x/[email protected]#modifying-and-building-a-new-schema on how to use "stock" bids-validator with a custom schema. (attn @TheChymera)

DONEs

• [x] Add (at least one) .md with the section for the added new modality/datatype
• [x] bids-standard/legacy-validator#1800

TODOs

• [ ] Please ensure your name is credited on our Contributors appendix. To add your name, please edit our Contributors wiki and add your name with the type of contribution. For assistance, please tag @bids-standard/maintainers.

• [ ] After opening the PR, our continuous integration services will automatically check your contribution for formatting errors and render a preview of the BIDS specification with your changes. To see the checks and preview, scroll down and click on the show all checks link. From the list, select the Details link of the ci/circleci: build_docs artifact check to see the preview of the BIDS specification.

• [x] Add instructions here on how to run new bids-validator using schema in this PR

  • Currently there is ongoing effort in https://github.com/bids-standard/bids-examples/pull/435 to assess the state of deno based validator against examples, then bids-standard/legacy-validator#1798 is the first one trying it on whitelisted set of packages, and I think we should create a helper action for that : https://github.com/bids-standard/bids-validator/issues/1931

• [ ] bids-validator changes needed

  • [ ] Add tricky use cases where validator should fail under https://github.com/bids-standard/bids-validator/tree/main/tests/data

• [ ] Populate schema with specifications from the google doc ...

  • [x] The majority of changes: https://github.com/bids-standard/bids-specification/pull/1826
  • [ ] Following @ree-gupta failed validation - tune up check for multiple electrode files to apply only to EEG, and iEEG modalities
  • [ ] Extend schema/rules/checks with checks specific to this BEP032 (from experiences with data conversion eg by @CodyCBakerPhD on @mvmdmlab data)
  • [ ] Similarly to iEEG specify <extension> and get a table of extensions (nwb and nix) and check if schema encodes that only one is allowed

• [x] Further markdown description: @Peyman-N is working on a PR

• [ ] Define enums of coord spaces to be added -- some image based, some ad-hoc

• [ ] Add and reference here PR on bids-examples adding sample dandisets : https://github.com/bids-standard/bids-examples/pull/430 (@robertoostenveld )

• [ ] Add CI action (likely github) to run bids-validator on sample datasets and this modified schema (@yarikoptic)

• [x] Decide on "contours" specification

  • decided to postpone
  • attempt to potentially generalize across modalities: #2013

• [ ] type column for electrodes

  • [ ] seek to replace hardcoded table in iEEG with a macro (@yarikoptic)
  • [x] add our new enums to src/schema/objects/enums.yaml and then within src/schema/objects/columns.yaml (@ree-gupta )
  • [ ] depending on decision in iEEG potentially add similar table in our [ie]cephys/ section

• [x] Share/use examples of real datasets

  • WiP: https://dandiarchive.org/dandiset/001470 AKA https://github.com/dandisets/001470 by @CodyCBakerPhD from @mvdm lab

Issues this PR would likely to address

• Fixes bids-standard/legacy-validator#1375

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

• bids-standard/legacy-validator#1634
• bids-standard/legacy-validator#1481

Other issues which relate but not in scope here and provided for reference/backreference

• bids-standard/legacy-validator#1669
• bids-standard/legacy-validator#1133
• bids-standard/legacy-validator#1165
• bids-standard/legacy-validator#197
• https://github.com/bids-standard/bids-2-devel/issues/54

Spreadsheet with correspondence to ProbeInterface: https://docs.google.com/spreadsheets/d/1O0bZzD-n4MjR68r1GlcH3d2JLXBLAU1PfsDceD3IPeo/edit?usp=sharing

steps to use bids-validator-deno against this modified schema

cd ../bids-specification-bep032
cd tools/schemacode/
uv venv
source .venv/bin/activate
uv pip install -e .
bst export > schema-bep032.json  # From bep032 branch
uv pip install bids-validator-deno
rehash  # zsh specific
python -m http.server 8899 &
bids-validator-deno --schema http://0.0.0.0:8899/schema-bep032.json /home/yoh/proj/bids/bids-examples/asl001/ --ignoreNiftiHeaders  # or localhost on windows

[TheChymera]

Also relevant if you'd like to comment. → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIzHGpUU

[TheChymera]

Also I forgot to link to this when I wrote it → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIGPAMOw

[Remi-Gau]

Random suggestion: maybe having a markdown file in the source tree and the mkdoc config so you can at least use a macro to render the filename template.

The HTML version of this could then be reviewed by people still working on the google doc: may help if you don't read BIDS schema yaml fluently.

[Remi-Gau]

Did a few minor fix to pacify pre-commit and make sure the HTML page rendered.

Also check the top message of this PR: https://github.com/bids-standard/bids-specification/pull/1705#issue-2134603705

I added some github admonitions that need tweaking so people know where this BEP is discussed.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.92%. Comparing base (1cb92eb) to head (73012b3). Report is 36 commits behind head on master.

:exclamation: Current head 73012b3 differs from pull request most recent head a46e439

Please upload reports for the commit a46e439 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1705      +/-   ##
==========================================
+ Coverage   87.79%   87.92%   +0.13%     
==========================================
  Files          16       16              
  Lines        1360     1375      +15     
==========================================
+ Hits         1194     1209      +15     
  Misses        166      166              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.