cbrnr
ICA :: Enable Auto-Labelling using ICALabel
The MNE-ICALabel package enables auto-labelling of ICA components based using a pretrained NN. Their output is a (label, probability) pair for each component, making it easier to include/exclude components - researchers don't need to make manual judgements per component. Instead I can decide to ignore anything that was labelled as some uninteresting label (see for example this article where they exclude any component that didn't get a brain label with p >= 80%).
I think it could be nice to add an "auto-label ICA" workflow: after fitting an ICA, the user can opt to use ICALabel to label their data, and include/exclude components based on the ICALabel's NN output.
I started working on something like this with @yuvishap, who added a "Label ICA" option under the "Tools" menu:
This opens a modal that presents the ICALabel labels, and enables modifying the state of the underlying ICA model:
Importantly, the user can still manually examine each component separately by double-clicking on the component:
Note: The exclusion of component in the table will be synchronized with the
ica.exclude state, so when user excludes a component on the table, it will be greyed out in the manual plot view and vice versa.
This is still a work in progress but shows there could be a working POC soon. Caveat: this still requires some manual actions by the user. A "completely automated workflow" (like the article above uses) can be implemented by having the user specify in advance what thresholds they want to apply to each type of label. However, we're not sure if people would really consider never looking at the ICA components and give complete control to a NN. We're open to discussion on the relevance of this user journey.
Technical note:
To use mne-icalabel, we need to include the following dependencies in the installation flow:
- mne-icalabel >= 0.7.0
- torch >= 2.0.0
I’ve added them locally in my development environment, but I’d appreciate your insights on whether these additions might introduce complications down the line based on your experience @cbrnr
Hi @JonNir1 and @Yuvishap! This is really great! Coincidentally, @SchellanderF and I have been thinking about integrating ICALabel into MNELAB ourselves. We haven't started implementing it yet, but we already have some ideas and concepts. However, since you have a working implementation, we should probably join forces, WDYT?
Regarding your technical question about adding dependencies, I think adding MNE-ICALabel is fine, because it is very lightweight. However, I would be hesitant to add PyTorch, which is really large and would increase the size of the binaries by a lot. Luckily, I don't think that this is necessary, as the ICLabel classifier is also available in the ONNX format, which only requires the onnxruntime package AFAIK.
Looking forward to discussing this topic with you in more detail! (Also CCing @hoechenberger as one of the original contributors, as he might also be interested in this feature.)
Hey @cbrnr, great to see that this idea was already on your roadmap - joining forces definitely makes sense!
I have a couple of questions regarding potential collaboration. We’d love to hear your thoughts and understand your preferred development workflow for this repo:
- Would you prefer that we take on the implementation work with your guidance and review, or would you rather split the implementation efforts between us?
- Do you have a timeline in mind for releasing the ICALabel feature?
Thoughts on PyTorch vs onnxruntime I completely understand your point regarding binary size. Since the app uses the model purely for inference, relying on the full PyTorch stack may indeed be unnecessary overhead. While looking into the implementation in mne-icalabel, I noticed that the project exposes two different API layers for labeling:
- Lower-level API: This allows selecting the backend explicitly, but defaults to PyTorch when none is provided. It returns only the raw class probabilities (no label mapping).
- Higher-level API: This handles input validation and the label-to-string mapping, but does not allow configuring the backend. It calls the function above, which means it always uses PyTorch.
I wasn’t able to find an example of using the higher-level API with the ONNX backend, so the tradeoff seems to be:
| Using Lower Level API | Using Higher Level API | |
|---|---|---|
| Pros | Configurable backend (supports ONNX) | Built-in validation & label mapping; future-proof if upstream changes |
| Cons | We must re-implement validation + label mapping (risk of divergence over time) | Hard-coded to PyTorch |
Another option would be to open an issue in mne-icalabel and request configurable backend support at a higher level in the API. This would provide a cleaner, more maintainable solution, but it would also introduce dependency on their release timeline and could slow down our development.
I’m happy to discuss this more and explore alternative approaches if you see a better path. Thanks, and looking forward to collaborating!
cc @JonNir1
Since @SchellanderF and I have already worked on this feature for a while now, I'd prefer that we share our implementation in a PR for you to provide feedback. I'm sure there will be a lot of things that need to be discussed (and eventually changed), so there will be opportunities for you to contribute code. Would this be OK for you?
Regarding the backends, mne-icalabel automatically uses onnxruntime if installed (and if pytorch is not installed). The only downside is that their main mne_icalabel.label_components() function only returns the probability for the most likely class, but we would like to include all probabilities in the dialog. However, we can use their internal mne_icalabel.iclabel.label_components.iclabel_label_components() function to get the probabilities for all seven classes. In addition to a dialog, we are also planning to include the most likely label directly in the components plot.
We've also been thinking about integrating the classifier directly into MNELAB, so we would not have to add the mne-icalabel dependency (this is not really complicated at all). Also, it might even be possible to get rid of onnxruntime by implementing the operations required to apply the classifier with NumPy, but this is still to be decided.
Re: I'd prefer that we share our implementation in a PR for you to provide feedback. I'm sure there will be a lot of things that need to be discussed (and eventually changed), so there will be opportunities for you to contribute code. Would this be OK for you?
Yes, of course. Happy to review the PR, and we’d also love to contribute code and get more involved