bids-validator

bids-validator copied to clipboard

Is there any relationship between bids-validator version and BIDS specification version? For example, I am preparing a BIDS dataset according to the latest BIDS specification (to the best of my ability) using "Brain Imaging Data Structure v1.6.0". Therefore, I am planning to mark BIDSVersion in our dataset_description.json to version v1.6.0.

However, I see v1.7.1 displayed from bids-validator

bids-validator --version
1.7.1

If my BIDS dataset passes the bids-validator v1.7.1, does it mean my BIDS dataset is "compatible" to BIDS v1.7.1?

Another related question to do this is that.. since BIDS specification has evolved over time, instead of showing if my dataset passes validation or not (for which BIDS specification version!?), maybe it could show which BIDS versions my dataset passes? For example, the output from bids-validator could be something like..

$ bids-validator mydataset
...
BIDS 1.4.1 PASS
BIDS 1.4.4 PASS
BIDS 1.5.0 FAIL
BIDS 1.6.0 FAIL

... basically showing which versions of BIDS specification my dataset passes so that I can correctly set the BIDSVersion field. I think this requires bids-validator to maintain sets of all checks performed by all previous versions of bids-validator..

Unfortunately, no. There's no relation between validator versions and specification versions.

Also, the spec is intended to be backwards compatible, so a valid 1.0.0 dataset is a valid 1.6.0 dataset.

The invalidation of a dataset by validator updates is due to increased coverage of the spec, and occasionally due to clarification of an ambiguous rule.

Once the spec is fully schematized and the validator works from the schema, we should be able to have versions of the schema that correspond to versions of the spec that would permit something like what you suggest at the end.

The invalidation of a dataset by validator updates is due to increased coverage of the spec,

This depends on the use-case, but for me, this means that the spec is not backward compatible. I ran into so many cases with older datasets published in OpenNeuro that (probably) passed BIDS validator a long time ago but no longer passes the latest BIDS validator. When I run the latest BIDS Apps, they simply fail by complaining about missing fields in the sidecar, or incompatible file structure altogether (for fmap), etc.

If BIDS apps could release each new version with supported BIDS specification versions, then I could try to run the older version of the App that supported the older BIDS spec version that matches the dataset's actual BIDS spec version.

For example.. let's say I downloaded a dataset from OpenNeuro that says BIDSVersion: "v1.4.0" (probably that was the latest version of the spec when this dataset was published) If bids-validator could enumerate all BIDS spec versions with PASS/FAIL on each version, then, I might find that this dataset actually passes the BIDS spec version all the way to v1.5.0. Then, if fmriprep published release notes staying the minimum required BIDS spec version, I could look for the latest version that supported spec v1.5.0 and run that version of fmriprep.

The invalidation of a dataset by validator updates is due to increased coverage of the spec,

This depends on the use-case, but for me, this means that the spec is not backward compatible.

The validator is not the spec. It is an incomplete implementation of the spec. As the validator becomes more complete, datasets that relied (generally implicitly) on its incompleteness will sometimes fail to validate. This doesn't make the spec text backwards incompatible.

When I run the latest BIDS Apps, they simply fail by complaining about missing fields in the sidecar,

An app may require some optional metadata. The spec intends to be broad, but apps sometimes cannot work without a particular piece of metadata.

or incompatible file structure altogether (for fmap), etc.

Can you give examples? The only breaks I can readily bring to mind are things like requiring the authors field in dataset description to be a list (the type had been unspecified, so some datasets had a comma-separated string). I don't know of any changes to file structures.

If BIDS apps could release each new version with supported BIDS specification versions, then I could try to run the older version of the App that supported the older BIDS spec version that matches the dataset's actual BIDS spec version.

This seems like a reasonable thing for apps to declare, but I'm not sure how much it's likely to change over time.

Also in light of the "variety" of BIDSVersion found on openneuro, I would take that piece of metadata with a pinch of salt.

See a listing of the versions found on openneuro (many of them do not match any exisiting BIDS version) https://github.com/bids-standard/bids-specification/issues/1469