moabb
moabb copied to clipboard
NeuroTechX
Consistency of session/run description of diferent datasets
Hi everyone,
would it be possible to name sessions/runs of datasets more consistently?
For example, the dataset BNCI2014001 comes as session: session_T and session_E with run: run_0 to run_5,
The dataset Schirrmeister2017 comes as session: session_1 with run: train and test.
Would it be possible to always get sessions corresponding to train (and valid and eval if they exist) and run as an enumeration of different times of recording or something similar?
It would then be clearer to the user what he can expect by fetching a dataset through MOABB, and it would also simplify switching from one dataset to another.
Yes, this is very good point. It is also problematic when aggregating the results of an Evaluation.
I'll try to push a new minor version this week to update this during the braindecode sprint. Is there a related issue in braindecode so I could mention it when it is done?
Great, thanks @sylvchev ! Among many other things, this was mentioned in https://github.com/braindecode/braindecode/issues/189#issuecomment-864914543.
I think the usual interpretation of session in a BCI context would be a recording session on one day. E.g. I would expect session 1 and session 2 to be on different days. Having different sessions for train, validation, eval would also create issues in using e.g. CrossSessionEvaluation, insofar I agree the sessions as defined in BNCI2014001 do not make sense.
Having the train/validation information on run-level would make more sense imo. For MOABB internally that info is not really relevant at the moment I guess, as we only use labelled data and train / validation folds are created unrelated to the train / validation setting of the underlying dataset.
@jsosulski I agree with you but I think that @gemeinl is refereing to the fact that in some datasets the session name are like session_T and session_E (for BNCI2014001) and in others it is more like session_1 and so on.