Feature: Support for file-like objects for EDF/BDF/GDF files

[szz-dvl]

Reference issue (if any)

Feature #8976.

What does this implement/fix?

Added support for file-like objects for EDF/BDF/GDF files.

Additional information

Until today, mne was checking for the file extension to validate file types, we may not have a file name from a binary object, so, if a binary object (file-like) is provided we trust the user (depending on what API function he is using: read_raw_edf, read_raw_gdf, read_raw_bdf). A new enumeration has been declared to propagate the file type to underlying methods:

class FileType(Enum):
    """Enumeration to differentiate files when the extension is not known."""

    GDF = 1
    EDF = 2
    BDF = 3

In the function _read_edf_header a new exception is raised if the header can be properly parsed, informing the user that the file provided is not valid:

try:
     header_nbytes = int(_edf_str(fid.read(8)))
except ValueError:
    raise Exception(
        f"Bad {'EDF' if file_type is FileType.EDF else 'BDF'} file provided."
    )

In the function _read_gdf_header, a new exception is raised when the version of the document can't be properly parsed, informing the user that the file provided is not valid:

try:
    version = fid.read(8).decode()
    edf_info["type"] = edf_info["subtype"] = version[:3]
    edf_info["number"] = float(version[4:])
except ValueError:
    raise Exception("Bad GDF file provided.")        

To accommodate this changes the class RawEDF class has been duplicated (RawBDF) to differentiate between BDF and EDF files when the extension is not known. To avoid changes in the file mne-python/mne/io/tests/test_raw.py.

Additionally 4 new tests have been added to the file mne-python/mne/io/edf/tests/test_edf.py and 2 new tests have been added to the file mne-python/mne/io/edf/tests/test_gdf.py, those tests will open a valid file and check for the channels, and open an invalid file and expect an error. For GDF I'm using an empty file (I don't know where to get bad GDF files).

I've created a helper function to open GDF and EDF files under mne-python/mne/_edf/open.py.

To support in memory file-like objects I'm using the workaround mentioned here, the function has been placed in "fixes.py". The calls to numpy.fromfile have then been replaced by calls to the new function read_from_file_or_buffer. The implementation is as follows:

def read_from_file_or_buffer(
    file: str | bytes | os.PathLike | io.IOBase, dtype: numpy.typing.DTypeLike = float, count: int = -1
):
    """numpy.fromfile() wrapper, handling io.BytesIO file-like streams.

    Numpy requires open files to be actual files on disk, i.e., must support
    file.fileno(), so it fails with file-like streams such as io.BytesIO().

    If numpy.fromfile() fails due to no file.fileno() support, this wrapper
    reads the required bytes from file and redirects the call to
    numpy.frombuffer().

    See https://github.com/numpy/numpy/issues/2230
    """
    try:
        return numpy.fromfile(file, dtype=dtype, count=count)
    except io.UnsupportedOperation as e:
        if not (e.args and e.args[0] == "fileno" and isinstance(file, io.IOBase)):
            raise  # Nothing I can do about it
        dtype = numpy.dtype(dtype)
        buffer = file.read(dtype.itemsize * count)
        return numpy.frombuffer(buffer, dtype=dtype, count=count)
        

[welcome[bot]]

Hello! 👋 Thanks for opening your first pull request here! ❤️ We will try to get back to you soon. 🚴

[szz-dvl]

@drammock The test preprocessing/tests/test_fine_cal.py::test_compute_fine_cal is failing in the pipeline, but I don't know how could I broke this one, surprisingly in my machine is failing for kind VectorView, but in the pipeline fails for kind TRIUX, I don't know how to proceed here...

Thanks!

[drammock]

preprocessing/tests/test_fine_cal.py::test_compute_fine_cal is failing in the pipeline, but I don't know how could I broke this one, surprisingly in my machine is failing for kind VectorView, but in the pipeline fails for kind TRIUX, I don't know how to proceed here

ignore, those are being fixed elsewhere. when the fixes are in main we can merge main here and the CIs should pass.

[szz-dvl]

@drammock https://github.com/mne-tools/mne-python/actions/runs/14239287761/job/39937221292?pr=13156 the old pipeline is still failing, this time however is numpy.typing who is missing, most probably due to an old numpy version, how would you like to tackle this one? must I remove the type annotation?

[drammock]

@drammock https://github.com/mne-tools/mne-python/actions/runs/14239287761/job/39937221292?pr=13156 the old pipeline is still failing, this time however is numpy.typing who is missing, most probably due to an old numpy version, how would you like to tackle this one? must I remove the type annotation?

Locally when I install numpy 1.25.2 (same as the failing CI) then this works

import numpy.typing
numpy.typing
# <module 'numpy.typing' from '/opt/mambaforge/envs/tst/lib/python3.11/site-packages/numpy/typing/__init__.py'>

but this fails

import numpy as np
np.typing
# AttributeError: module 'numpy' has no attribute 'typing'. Did you mean: '_typing'?

Which is a "lazy loading" problem (typing submod not imported in module __init__). Solution should be to explicitly import the typing submodule wherever it's needed (i.e., in the mne/fixes.py file)

[szz-dvl]

@larsoner Pushing the changes

[szz-dvl]

@larsoner I took a look at the failing tests, but I don't know the reason they are failing, do you have any clue?

Thanks!

[drammock]

@larsoner I took a look at the failing tests, but I don't know the reason they are failing, do you have any clue?

Thanks!

CI errors are unrelated, tracking them at #13221

[larsoner]

Sorry we forgot about this one @szz-dvl -- feel free to ping us if it's time for a (re-)review and it has been a couple of weeks! I pushed a merge with upstream/main and a couple of tiny cosmetic changes, otherwise looks like you addressed the comments above so I'll mark for merge-when-green. Thanks in advance @szz-dvl !

[szz-dvl]

Hey @larsoner I completely forgot about this one, Please give me tomorrow morning to review everything and I think we will be ready to go.

Thanks to you!

[szz-dvl]

@larsoner I'm not able to see what is going wrong with docs and lincheck, any clue?

Thanks

[larsoner]

It has to do with CircleCI build permissions... I pushed an empty commit that should come back green. I'll mark for merge-when-green, thanks in advance @szz-dvl !

[welcome[bot]]

🎉 Congrats on merging your first pull request! 🥳 Looking forward to seeing more from you in the future! 💪