Update indexer to treat .zarr as files not dirs
Microscopy file formats were recently updated in the standard to include ome-zarr, which are stored in a directory suffixed with .ome.zarr.
https://bids-specification.readthedocs.io/en/stable/modality-specific-files/microscopy.html#file-formats
This however doesn't work with pybids currently, since the pybids just indexes it as a directory.. This can cause issues if there is an ome.zarr folder with real data, since there are a large number of files in that folder which slow the indexing to a crawl..
This adds a quick and simple (albeit hardcoded) fix to ensure .zarr directories get indexed as files instead of directories. This enables us to get .ome.zarr microscopy files, and also prevents needless indexing of .zarr directories that contain a large number of files.
This solves my problem as I can now use ome.zarr in bids, seemingly without any negative side effects (?) - a more configurable solution would seem nicer of course, but not sure how often this would come up in the future (ie bids "files" that are actually directories), so not sure it would be worth the added config set-up?
Thoughts? Pinging @satra as someone in the BIDS community that was also involved in the ome-zarr standard.
Note: .ome.zarr does not yet seem to be in the pybids config, so using validate=False in the layout to test things..
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 89.80%. Comparing base (
6aaa5f2) to head (d556233).
Additional details and impacted files
@@ Coverage Diff @@
## master #1046 +/- ##
=======================================
Coverage 89.79% 89.80%
=======================================
Files 63 63
Lines 7175 7178 +3
Branches 1372 1374 +2
=======================================
+ Hits 6443 6446 +3
Misses 532 532
Partials 200 200
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Agreed a more configurable solution would be nice. But I think that can wait until we depend on the BIDS schema, which does indicate extensions of data "files" that are actually directories.