mne-bids-pipeline icon indicating copy to clipboard operation
mne-bids-pipeline copied to clipboard

Any thoughts about nipype?

Open bloyl opened this issue 5 years ago • 6 comments

This seems like an interesting and potentially useful project.

I'm curious if there was any thought to making a nipype (https://nipype.readthedocs.io/en/latest/) type interfaces?

I don't use nipype very often but I have found it useful in the past to standardize some large batch processing. Just a thought.

bloyl avatar Dec 12 '19 18:12 bloyl

And maybe https://neuropycon.github.io/ephypype/ to make it easy to use nipype for these tasks?

larsoner avatar Dec 12 '19 18:12 larsoner

It would be great if you can try this on CTF data to identify potential pain points in using this. I suggested the same to @AdoNunes

as for neuropycon, give it a try but maybe joblibs does the job already and you don't need caching.

jasmainak avatar Dec 12 '19 20:12 jasmainak

what would be the benefit of nipype? our current approach is pure python so no need to mix languages and packages. I find our approach simple in understanding what we pass as input and get as output

agramfort avatar Dec 12 '19 22:12 agramfort

What I think nipype has to offer is scalability, flexibility and sharing, but that comes at the expense of some of the simplicity and clarity.

Some of the things I think are nice about nipype are its

  1. workflow visualization/diagrams (see the middle bottom of https://miykael.github.io/nipype_tutorial/notebooks/example_preprocessing.html)
  2. workflow hotstarts in the event of errors etc.
  3. optional overwriting etc. controlled at the workflow level.
  4. sharability: if the nodes here were in nipype I could just use them.
  5. its a robust pipelining tool - why invent another one?

Also it seems like someone is supporting the mne-c and mne-pthon function calls (https://github.com/nipy/nipype/issues/3032)

Like i said before I don't really use nipype but it is something that i think i would use if the barrier to entry was alittle bit lower.

bloyl avatar Dec 13 '19 02:12 bloyl

I stay open but I feel this is premature here. We have something that works and that needs to be consolidated. Once we have a stable code/solution why not looking into different strategies.

agramfort avatar Dec 13 '19 14:12 agramfort

has anybody looked into doit: https://pydoit.org/ ?

jasmainak avatar Oct 07 '20 16:10 jasmainak

I think we have decided to use joblib caching at this point so I'll close

larsoner avatar Nov 08 '22 21:11 larsoner