fmriprep icon indicating copy to clipboard operation
fmriprep copied to clipboard
verified publisher icon nipreps

fMRIPrep 21.0.2 hangs with no errors

Open ypicard opened this issue 3 years ago • 1 comments

What happened?

Running fMRIPrep 21.0.2 inside Docker, it stalls without any useful information or error on why. It is very hard to know where to look for a solution.

What command did you use?

The COMMAND in the Docker container is the following:

`/opt/conda/bin/fmriprep /data /out participant --debug all --output-layout legacy --aroma-melodic-dimensionality=-200 --bold2t1w-dof=6 --bold2t1w-init=register --dvars-spike-threshold=1.5 --fd-spike-threshold=0.5 --fs-no-reconall --ignore fieldmaps slicetiming --mem_mb=16384 --n_cpus=16 --output-spaces anat --skull-strip-t1w=force --skull-strip-template=OASIS30ANTs --use-syn-sdc --verbose --notrack --participant-label 01032 --bids-filter-file /out/filter.json -w /scratch`

What version of fMRIPrep are you running?

21.0.2

How are you running fMRIPrep?

Docker

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

2022-07-23T09:11:28.172778788Z   [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 16.18/16.38, Free processors: 15/16.
2022-07-23T09:11:28.172781944Z                      Currently running:
2022-07-23T09:11:28.172783963Z                        * fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.extract_field
2022-07-23T09:11:29.627863766Z 220723-09:11:29,627 nipype.workflow INFO:
2022-07-23T09:11:29.627887546Z   [Node] Finished "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.extract_field".
2022-07-23T09:11:30.173299382Z 220723-09:11:30,173 nipype.workflow INFO:
2022-07-23T09:11:30.173313161Z   [Job 293] Completed (fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.extract_field).
2022-07-23T09:11:30.175661498Z 220723-09:11:30,175 nipype.workflow INFO:
2022-07-23T09:11:30.175674528Z   [MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 16.38/16.38, Free processors: 16/16.
2022-07-23T09:11:30.253552989Z 220723-09:11:30,253 nipype.workflow INFO:
2022-07-23T09:11:30.253586201Z   [Node] Setting-up "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_field" in "/scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/wf_auto_00000/zooms_field".
2022-07-23T09:11:30.258282918Z 220723-09:11:30,258 nipype.workflow INFO:
2022-07-23T09:11:30.258296250Z   [Node] Running "zooms_field" ("niworkflows.interfaces.fixes.FixHeaderApplyTransforms"), a CommandLine Interface with command:
2022-07-23T09:11:30.258299598Z antsApplyTransforms -u float --default-value 0 --float 0 --input /scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/wf_auto_00000/extract_field/fmap_syn0Warp_Hz.nii.gz --interpolation BSpline --output fmap_syn0Warp_Hz_trans.nii.gz --reference-image /scratch/fmriprep_wf/single_subject_01032_wf/syn_preprocessing_auto_00000/deob_epi/clipped_plumb.nii.gz --transform identity
2022-07-23T09:11:31.938622218Z 220723-09:11:31,938 nipype.workflow INFO:
2022-07-23T09:11:31.938644873Z   [Node] Finished "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_field".
2022-07-23T09:11:32.175309548Z 220723-09:11:32,175 nipype.workflow INFO:
2022-07-23T09:11:32.175322672Z   [Job 294] Completed (fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_field).
2022-07-23T09:11:32.177652478Z 220723-09:11:32,177 nipype.workflow INFO:
2022-07-23T09:11:32.177664080Z   [MultiProc] Running 0 tasks, and 2 jobs ready. Free memory (GB): 16.38/16.38, Free processors: 16/16.
2022-07-23T09:11:32.256787905Z 220723-09:11:32,256 nipype.workflow INFO:
2022-07-23T09:11:32.256810570Z   [Node] Setting-up "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_bmask" in "/scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/wf_auto_00000/zooms_bmask".
2022-07-23T09:11:32.259463541Z 220723-09:11:32,259 nipype.workflow INFO:
2022-07-23T09:11:32.259476361Z   [Node] Running "zooms_bmask" ("niworkflows.interfaces.fixes.FixHeaderApplyTransforms"), a CommandLine Interface with command:
2022-07-23T09:11:32.259479596Z antsApplyTransforms -u uchar --default-value 0 --float 0 --input /scratch/fmriprep_wf/single_subject_01032_wf/syn_preprocessing_auto_00000/mask2epi/09_relabel_wm_mask_xform_trans_uint8.nii.gz --interpolation MultiLabel --output 09_relabel_wm_mask_xform_trans_uint8_trans.nii.gz --reference-image /scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/wf_auto_00000/zooms_field/fmap_syn0Warp_Hz_trans.nii.gz --transform identity
2022-07-23T09:11:32.288486314Z 220723-09:11:32,288 nipype.workflow INFO:
2022-07-23T09:11:32.288505727Z   [Node] Setting-up "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.bs_filter" in "/scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/wf_auto_00000/bs_filter".
2022-07-23T09:11:32.290296010Z 220723-09:11:32,290 nipype.workflow INFO:
2022-07-23T09:11:32.290309323Z   [Node] Running "bs_filter" ("sdcflows.interfaces.bspline.BSplineApprox")
2022-07-23T09:11:34.178880299Z 220723-09:11:34,178 nipype.workflow INFO:
2022-07-23T09:11:34.178911940Z   [MultiProc] Running 2 tasks, and 0 jobs ready. Free memory (GB): 15.98/16.38, Free processors: 7/16.
2022-07-23T09:11:34.178925712Z                      Currently running:
2022-07-23T09:11:34.178927823Z                        * fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.bs_filter
2022-07-23T09:11:34.178929764Z                        * fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_bmask
2022-07-23T09:11:44.346291139Z 220723-09:11:44,345 nipype.workflow INFO:
2022-07-23T09:11:44.346316038Z   [Node] Finished "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_bmask".
2022-07-23T09:11:46.189561076Z 220723-09:11:46,189 nipype.workflow INFO:
2022-07-23T09:11:46.189589432Z   [Job 295] Completed (fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.zooms_bmask).
2022-07-23T09:11:46.192025194Z 220723-09:11:46,191 nipype.workflow INFO:
2022-07-23T09:11:46.192039438Z   [MultiProc] Running 1 tasks, and 1 jobs ready. Free memory (GB): 16.18/16.38, Free processors: 8/16.
2022-07-23T09:11:46.192042570Z                      Currently running:
2022-07-23T09:11:46.192044750Z                        * fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.bs_filter
2022-07-23T09:11:46.286600265Z 220723-09:11:46,286 nipype.workflow INFO:
2022-07-23T09:11:46.286627676Z   [Node] Setting-up "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.out_merge_fmap_mask" in "/scratch/fmriprep_wf/single_subject_01032_wf/fmap_preproc_wf/out_merge_fmap_mask".
2022-07-23T09:11:46.288348872Z 220723-09:11:46,288 nipype.workflow INFO:
2022-07-23T09:11:46.288362255Z   [Node] Running "out_merge_fmap_mask" ("nipype.interfaces.utility.base.Merge")
2022-07-23T09:11:46.289898408Z 220723-09:11:46,289 nipype.workflow INFO:
2022-07-23T09:11:46.289910835Z   [Node] Finished "fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.out_merge_fmap_mask".
2022-07-23T09:11:48.191607256Z 220723-09:11:48,191 nipype.workflow INFO:
2022-07-23T09:11:48.191634179Z   [Job 297] Completed (fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.out_merge_fmap_mask).
2022-07-23T09:11:48.193943410Z 220723-09:11:48,193 nipype.workflow INFO:
2022-07-23T09:11:48.193955219Z   [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 16.18/16.38, Free processors: 8/16.
2022-07-23T09:11:48.193958123Z                      Currently running:
2022-07-23T09:11:48.193960142Z                        * fmriprep_wf.single_subject_01032_wf.fmap_preproc_wf.wf_auto_00000.bs_filter



### Additional information / screenshots

_No response_

ypicard avatar Jul 23 '22 13:07 ypicard

It seems you are hitting a wall with the --use-syn-sdc pathway of the workflow - I would avoid using that flag with your data for the time being.

If you have any data that you can share with us (T1w + BOLD), it will help us tremendously to improve the robustness of the SyN implementation

mgxd avatar Aug 01 '22 13:08 mgxd

Debugging this issue in nipreps/sdcflows#290. Will leave it open here so people with the same issue can find this.

effigies avatar Oct 19 '22 14:10 effigies

Fixed in 22.1.0.

effigies avatar Dec 15 '22 16:12 effigies