wfdb-python icon indicating copy to clipboard operation
wfdb-python copied to clipboard

Maybe wfdb should not read .{dat,hea,...}, instead it should be possible to directly give file contents to wfdb.

Open Dubrzr opened this issue 8 years ago • 2 comments

The fact that wfdb read files for you and that you don't have to open the file is nice, but when your files are not on the local filesystem it is much more complicated to open those files with wfdb. For example: Let's say all my files are on Hadoop HDFS (which is actually the case), the only way to access them in python is using a library such as hdfs3 (https://github.com/dask/hdfs3):

with hdfs.open('/user/data/100.dat') as f:
    content = f.read()
# I am now forced by wfdb to write the contents to a temporary file in the local filesystem to then open it with wfdb...

I know it would be very annoying to modify the current implementation of wfdb but I have the feeling it would be better to let the user choose its own way to give .{dat, hea, ...} contents to wfdb. The way I see this:

  1. You have the wfdb-python library that only takes contents as input
  2. You have helper functions (downloads from physionet and read the contents for you)

Do you see a better solution?

Dubrzr avatar Sep 26 '17 14:09 Dubrzr

I might add to this that there is no way to spesify where .hea files to write to. Currently I can change the directory where .dat writes to by modifying the filename attribute of Record.

lkjell avatar Dec 08 '17 00:12 lkjell

For HDFS, one can always use the NFS gateway and mount the required HDFS path as a NFS folder. I'm sure there should be something similar for Object data store (AWS S3, GCS Bucket).

IMO a better solution would be the ability to write to an IO ByteStream instead. This would give users a memory object that could be managed however needed (HDFS, S3, GCP objects, etc).

If the maintainers agree, I could try and pitch in some dev time.

josemarfdc avatar Jul 06 '21 18:07 josemarfdc