mne-bids-pipeline
mne-bids-pipeline copied to clipboard
mne-tools
Extending the standard pipeline
Hi,
I have code that detects flux jumps, and I've managed to add it to the pipeline with a couple hacks. But it's clear that it's not really meant to be done.
There's a couple of things which make this difficult:
- Importing functions from other scripts, as
load_datafrom01-import_and_maxfilter.pyis not possible because of the leading digit. It might be fixable, but I couldn't figure out how. - The logging system is not designed to add in extra steps.
All of this is not really an issue, and the code runs just fine, but if extending the pipeline is part of the future plans I think there should be some modifications done. Is it part of the roadmap for mne-bids-pipeline?
Probably the biggest issue is the fact that the pipeline is split up with one file for each step, which means that the code is split up among multiple files, and things are pretty static as a result. If there could be a way to group the functions into a couple files, and then dynamically generate the pipeline scripts based on the config parameters, that could allow users to add in more steps easily, at the cost of making the inner workings less explicit.
Hello, making the Pipeline more versatile and allowing for custom code to be executed is definitely something we would love to do. Would you expect to have your code run within any of the existing steps, or could it be kept separate, plugged “between” steps?
shall we push the flux jump detection code into mne in an annotate_flux_jump function as discussed last time?
annotate_flux_jump function as discussed last time?
Absolutely YES to this one!
But it's also a good idea to discuss where we should provide hooks for custom code :)
shall we push the flux jump detection code into mne in an annotate_flux_jump function as discussed last time?
Can we talk about this during office hours?
A more fleshed out idea is in https://github.com/mne-tools/mne-bids-pipeline/issues/924#issuecomment-2065293402 so I'll close this in favor of discussion there
