Make it work for "inconsistent runs" (using ds001971)

[jasmainak]

To do

• [ ] make use of MNE-Python to handle duplicate events: https://github.com/mne-tools/mne-python/pull/6688
• [ ] make mne-study-template work for inconsitent runs

[jasmainak]

This PR should be merged after #40

[jasmainak]

I get:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~/Desktop/projects/github_repos/mne-study-template/05a-run_ica.py in <module>()
    171     parallel, run_func, _ = parallel_func(run_ica, n_jobs=config.N_JOBS)
    172     parallel(run_func(subject, session) for subject, session in
--> 173              itertools.product(config.subjects_list, config.sessions))

~/Desktop/projects/github_repos/mne-study-template/05a-run_ica.py in <genexpr>(.0)
    170 if config.use_ica:
    171     parallel, run_func, _ = parallel_func(run_ica, n_jobs=config.N_JOBS)
--> 172     parallel(run_func(subject, session) for subject, session in
    173              itertools.product(config.subjects_list, config.sessions))

~/Desktop/projects/github_repos/mne-study-template/05a-run_ica.py in run_ica(subject, session)
    136             ica.fit(epochs_for_ica, decim=config.ica_decim)
    137         else:
--> 138             ica.fit(epochs_for_ica, picks=picks, decim=config.ica_decim)
    139 
    140         print('  Fit %d components (explaining at least %0.1f%% of the'

</home/mainak/Desktop/projects/github_repos/mne-python/mne/externals/decorator.py:decorator-gen-394> in fit(self, inst, picks, start, stop, decim, reject, flat, tstep, reject_by_annotation, verbose)

~/Desktop/projects/github_repos/mne-python/mne/utils/_logging.py in wrapper(*args, **kwargs)
     87             with use_log_level(verbose_level):
     88                 return function(*args, **kwargs)
---> 89         return function(*args, **kwargs)
     90     return FunctionMaker.create(
     91         function, 'return decfunc(%(signature)s)',

~/Desktop/projects/github_repos/mne-python/mne/preprocessing/ica.py in fit(self, inst, picks, start, stop, decim, reject, flat, tstep, reject_by_annotation, verbose)
    472                           tstep, reject_by_annotation, verbose)
    473         elif isinstance(inst, BaseEpochs):
--> 474             self._fit_epochs(inst, picks, decim, verbose)
    475 
    476         # sort ICA components by explained variance

~/Desktop/projects/github_repos/mne-python/mne/preprocessing/ica.py in _fit_epochs(self, epochs, picks, decim, verbose)
    569         # more from _pre_whiten)
    570         data, self.pre_whitener_ = \
--> 571             self._pre_whiten(np.hstack(data), epochs.info, picks)
    572 
    573         self._fit(data, self.max_pca_components, 'epochs')

~/anaconda2/envs/mne/lib/python3.6/site-packages/numpy/core/shape_base.py in hstack(tup)
    286         return _nx.concatenate(arrs, 0)
    287     else:
--> 288         return _nx.concatenate(arrs, 1)
    289 
    290 

ValueError: need at least one array to concatenate

This is a bad error message from mne-python end because all the epochs were dropped and that's not what the error message says. Another easy fix @sappelhoff ?

I need to stop now. Can you take over @sappelhoff ?

[jasmainak]

@sappelhoff why is CircleCI failing here?

[jasmainak]

@sappelhoff can you make the CIs happy here?

[jasmainak]

@sappelhoff where did my commit disappear? :)

[sappelhoff]

@sappelhoff where did my commit disappear? :)

:scream: which commit?

[sappelhoff]

uff, got it: https://github.com/mne-tools/mne-study-template/commit/c28281b29e347e04c84b6139dd463e6e539ade98

no clue ~~how it disappeared ...~~ why it did not survive the rebase and force push

[sappelhoff]

so commit is back @jasmainak ... but now it's failing of course :-)

I'll add this to the project board

[jasmainak]

@sappelhoff don't add too much to the project board. It'll just be me and maybe one more person on this. And we're there for 4.5 days split over many projects ...

[sappelhoff]

@sappelhoff don't add too much to the project board. It'll just be me and maybe one more person on this. And we're there for 4.5 days split over many projects ...

so you are telling me I should not put the excess work from my GSoC on the project board? :smile:

sounds reasonable ... but it felt so nice

[jasmainak]

what did not get done in 3 months will not get done in 4 days :)

[hoechenberger]

I incorrectly referenced this PR in a different commit. Will start working on this one now.

[larsoner]

@jasmainak any interest in redoing this with latest main?

[jasmainak]

umm ... I'm not really using mne-bids-pipeline myself, so not my top priority ... I think this was more for @sappelhoff 's GSoC to figure out bugs and smooth them out.

[larsoner]

I have been using MNE-BIDS-Pipeline for inconsistent runs with some success, and opened #820 to discuss a somewhat related issue. Let's close this for now and open follow-up PRs as needed to address specific use cases.