moabb icon indicating copy to clipboard operation
moabb copied to clipboard

Adding ERPCore datasets

Open tahatt13 opened this issue 1 year ago • 2 comments

The purpose of this PR is to add ERPCore datasets Here is the link to the official website of the dataset : https://erpinfo.org/erp-core.

tahatt13 avatar Jun 17 '24 10:06 tahatt13

I'm not sure of the events I kept for ERP and LRP as Target and NonTarget. In addition, with the events i kept i get an error because the classes are unbalanced especially for ERP individual n 5, where only one one class is present.

tahatt13 avatar Jun 28 '24 09:06 tahatt13

@tahatt13 I tried downloading subject 1 but got an Http read timeout. Could you download it? Is it a problem with the server or with the url?

PierreGtch avatar Jul 08 '24 22:07 PierreGtch

I have tested to install the moabb version with the ERPCORE2021, and try to get the data but I get this warning : InsecureRequestWarning: Unverified HTTPS request is being made to host '[files.osf.io](http://files.osf.io/)'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings and this error: requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='[files.osf.io](http://files.osf.io/)', port=443): Read timed out. (read timeout=5)

sebVelut avatar Feb 14 '25 09:02 sebVelut

With the latest version, download works on my side But the local file path is strange:

Downloading data from 'https://files.osf.io/v1/resources/q6gwp/providers/osfstorage/600df65e75226b017d517f6d/?zip=' to file '/Users/Pierre.Guetschel/mne_data/MNE-erpcoreern2021-data/MNE-erpcoreern2021-data/v1/resources/q6gwp/providers/osfstorage/?zip='.

And the file can't be loaded:

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "/Users/Pierre.Guetschel/Projects/moabb/moabb/datasets/base.py", line 433, in get_data
    data[subject] = self._get_single_subject_data_using_cache(
  File "/Users/Pierre.Guetschel/Projects/moabb/moabb/datasets/base.py", line 527, in _get_single_subject_data_using_cache
    sessions_data = self._get_single_subject_data(subject)
  File "/Users/Pierre.Guetschel/Projects/moabb/moabb/datasets/erpcore2021.py", line 213, in _get_single_subject_data
    file_path = self.data_path(subject)[0]
  File "/Users/Pierre.Guetschel/Projects/moabb/moabb/datasets/erpcore2021.py", line 271, in data_path
    dataset_path = self.download_and_extract(path=path, force_update=force_update)
  File "/Users/Pierre.Guetschel/Projects/moabb/moabb/datasets/erpcore2021.py", line 324, in download_and_extract
    zip_ref = z.ZipFile(path_zip, "r")
  File "/Users/Pierre.Guetschel/miniforge3/envs/moabb/lib/python3.9/zipfile.py", line 1268, in __init__
    self._RealGetContents()
  File "/Users/Pierre.Guetschel/miniforge3/envs/moabb/lib/python3.9/zipfile.py", line 1335, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Also I get the same warning as @sebVelut :

InsecureRequestWarning: Unverified HTTPS request is being made to host 'files.osf.io'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings

PierreGtch avatar Feb 14 '25 09:02 PierreGtch

it is done @sebVelut and @PierreGtch, please review when everything it is green :)

bruAristimunha avatar Feb 25 '25 11:02 bruAristimunha

Hey @PierreGtch, @sebVelut and @sylvchev!

I would like to merge theses datasets to include into my benchmark, can we merge today?

bruAristimunha avatar Feb 26 '25 07:02 bruAristimunha

Ok @sebVelut, thanks, I will merge this when everything is green :)

bruAristimunha avatar Feb 26 '25 08:02 bruAristimunha

Thank you so much @tahatt13 🙏🏽

bruAristimunha avatar Feb 26 '25 10:02 bruAristimunha