wfdb-python icon indicating copy to clipboard operation
wfdb-python copied to clipboard

read records stored in AWS S3

Open entest-hai opened this issue 5 years ago • 2 comments

Can we add a function to read records stored in AWS S3 like Pandas?

entest-hai avatar Dec 15 '20 03:12 entest-hai

This issue is linked to this one : https://github.com/MIT-LCP/wfdb-python/issues/73

You should be able to open the file on any filesystem (local, remote S3, HDFS...) and read the contents of it using wfdb-python, instead of implementing a connection to these different filesystems directly into wfdb-python :)

Dubrzr avatar Dec 16 '20 11:12 Dubrzr

Hey guys thanks for the suggestion! I can get this to work by doing this so I'll make a pull request shortly:

>>> import s3fs
>>> fs = s3fs.S3FileSystem(anon=True)
>>> with fs.open('s3://BUCKET_NAME/100.hea') as f: print(f.readlines())
... 
[b'# unnecessary comment\n', b'100 2 360 650000\r\n', b'100.dat 212 200 11 1024 995 -22131 0 MLII\n',
b'\r\n', b'100.dat 212 200 11 1024 1011 20052 0 V5\r\n', b'# 69 M 1085 1629 x1\r\n',
b'# Aldomet, Inderal\r\n']

Lucas-Mc avatar Apr 27 '21 12:04 Lucas-Mc