What needs to be done to make xdf an official format for BIDS?

[cboulay]

Here is the relevant text from the BIDS BEP006: BIDS-EEG extension proposal.

While there are many file formats to store EEG data, there are 2 prefered formats in BIDS EEG: European data format (.edf) and BrainVision (.vhdr, .vmrk, .eeg) by Brain Products. There are also 3 additional ‘supported’ formats: EEGLAB set files (.set, .fdt), Neuroscan CNT (.cnt) and Biosemi (.bdf). The current selection is based on interoperability (one of the FAIR principles) between the main open source MEEG software, i.e. EEGLAB, FieldTrip (which supports SPM), Brainstorm and MNE Python. Note that most manufacturers’ software offers options to convert to .edf and original data format can be stored in the /sourcedata folder (see below).

Future versions of BIDS may extend this list of supported file formats (in particular for /derivatives). File formats for consideration MUST have open access documentation, MUST have open source implementation for both reading and writing in at least two programming languages and SHOULD be widely supported in multiple software packages. Other formats that may be considered in the future should have a clear added advantage over the existing formats and should have wide adoption in the BIDS-EEG community.

• [ ] Add xdf writer support for Python and Matlab. Should this be entirely native (i.e. pure Python) or using #13 (which should be made C)
• [ ] Add xdf support to MNE Python. Is python-neo the best route here?
• [ ] Add xdf support to Fieldtrip
• [x] FAIR F1 -> Making it easier in LabRecorder to give more useful and unique filenames.
• [ ] FAIR A2 "metadata are accessible, even when the data are no longer available." -> Need a tool to strip out data and leave only metadata.

Please volunteer to take on any of these items.

[sappelhoff]

+1!

regarding point 1, I think that a pure python implementation is preferable, because it is easier to understand for a larger audience (there are fewer people who can read and understand C / C++ code)

regarding point 2, please see: https://github.com/mne-tools/mne-python/issues/5180

[dmedine]

I said a long time ago that I would shove xdf into MNE. The issue there (if I understand correctly) is that if there are multiple streams, they all have to have the same sampling rate, first time-stamp, etc. This makes it labor intensive enough for me to avoid doing it.

However, if MNE is like EEGLab and it is content to grab only EEG and marker streams, then this might not be such a chore after all.

The BIDS people are coming to Brain Products in December. Hopefully I will also be there at that time and can discuss this with them personally.

[tstenner]

With #19 the plain python reader's speed is within an order of magnitude compared to a pure C++ implementation so there's no binary extension / Cython magic needed (same for Matlab).

[arnodelorme]

I think this is the route:

• Move away from the beta version (release stable version)
• Gather some support from manufacturers who support LSL
• Make another poll showing XDF is widely used (note that it is the 3rd most popular format on EEGLAB with 1153 downloads - so it is popular).
• Ensure the importer for MNE and Fieldtrip is functional and stable
• Have an extractor that can convert XDF events to BIDS compatible event tables

[DominiqueMakowski]

Sorry to revive this thread, but I am currently trying to upload data acquired with LSL / LabRecorder to OpenNeuro but it doesn't seem to like .xdf files... Was there by any chance any progress on this front?

[cboulay]

@DominiqueMakowski , not as far as I know. I think your best bet is to convert the XDF files to a BIDS format.

[DominiqueMakowski]

Assuming my xdfs contains bodily signals (ECG, RSP) and EEG (from muse), any wild guess about what would the best way to do that?

I am thinking to open the xdf files, separate the eeg from the physio, save the former as .edf and the later as .tsv.gz, but maybe there's something more straightforward

[cboulay]

Just replying to let you know that I'm not a BIDS expert and I think you'll get a better answer in a BIDS forum. Do you use EEG-BIDS with sidecar files? Do you use two parallel BIDS folders? I don't know.

[DominiqueMakowski]

Fair enough, I'm still new to this so I don't have a good grasp of the ecosystem yet Thanks for your help, though, looking forward to seeing how xdf evolves :)