mne-python icon indicating copy to clipboard operation
mne-python copied to clipboard

Artifact Subspace Reconstruction

Open DiGyt opened this issue 3 years ago • 17 comments

Reference issue

Fixes #7479.

What does this implement/fix?

Introduces Artifact Subspace Reconstruction for cleaning raw EEG.

Additional information

Sorry for the long waiting since my mention in issue #6290.

I wanted to make sure that this algorithm was as equivalent as possible to the original ASR implementation in MATLAB.

@nbara, first of all thank you very much for your implementation, this was a great thing to start with and I could many functions without any need to change. Due to your contribution to the code, I mentioned you as an author in the corresponding files (but without the E-Mail address). Are you okay with that or should I change it in some way?

Side note @nbara : Although your code looks really solid on a theoretical level, most of the operations converge from the original implementation. You are probably aware of most of the conceptual differences (using different functions or loops etc), but I also think I found smaller mistakes like dot products on matrices where actually the transposed matrices were required (I think this happened somewhere in asr_calibrate or one of its helper functions). Just wanted to tell you this, since I think this can change the data drastically...

Equivalence to the original

In this implementation, I made sure that asr_calibrate and asr_process are perfectly equivalent with the original ASR implementation. However there were several steps that introduce a slight divergence of the data. This is mostly due to different solvers used in MATLAB/Python. These steps are:

  • The moore-penrose pseudoinverse (due to a different solver in MATLAB)
  • Calculating the eigenvectors and eigenvalues (probably similar problem)
  • fitting an EEG distribution fit_eeg_distribution
  • extracting clean data windows clean_windows

Everything else in these functions should be entirely equivalent, leading to an average channel correlation of r=0.94-0.97 between original version and this implementation (as compared to r~0.0 with uncleaned data or the ASR implementation this was based on).

Concerning testing, I am not sure how we should optimally proceed. For now I test the correlation of cleaned data with data cleaned with the original ASR. This is obviously not perfect, as it's not a precise assertion and requires additional external data for testing (for now I added this test data directly to preprocessing/tests/data, assuming that it will be changed anyway).

riemannian ASR

The riemannian variant of Artifact Subspace Reconstruction is not yet available, as it requires pyriemann (and probably pymanopt) as dependencies and the current version of pyriemann has compatibility issues with the newer scipy versions (See alexandrebarachant/pyRiemann#94 and alexandrebarachant/pyRiemann#98). But as most people report no big difference between ASR/rASR anyway, this should be okay for now. However, I left a method parameter and notes to later adapt the riemannian variant more easily.

DiGyt avatar Apr 15 '21 17:04 DiGyt