nipreps
[B0FieldIdentifier] repeated between subjects makes fmriprep crash
What happened?
I tried using B0FieldIdentifier and B0FieldSource for susceptibility distortion correction. According to BIDS Spec, I've added
"B0FieldIdentifier": "phdiff_fmap0",
in .../fmap/...phasediff.json, and added
"B0FieldSource": "phdiff_fmap0",
in .../func/...bold.json for each subjects. I.e., I use the same identifier for all subjects.
When I tried run fmriprep for multiple participants at once, I got the error.
Is it required that this identifier MUST be different across subjects? Or are there chances to improve fmriprep such that we only need the identifier be unique within one subject?
What command did you use?
ocker run -it --rm -v bids:/data:ro -v fmriprep:/out nipreps/fmriprep:24.0.1 --notrack --fs-no-reconall /data /out participant
What version of fMRIPrep are you running?
24.0.1
How are you running fMRIPrep?
Docker
Is your data BIDS valid?
Yes
Are you reusing any previously computed results?
No
Please copy and paste any relevant log output.
Process Process-2:
Traceback (most recent call last):
File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/cli/workflow.py", line 126, in build_workflow
retval['workflow'] = init_fmriprep_wf()
^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 94, in init_fmriprep_wf
single_subject_wf = init_single_subject_wf(subject_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 515, in init_single_subject_wf
fmap_estimators, estimator_map = map_fieldmap_estimation(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 763, in map_fieldmap_estimation
fmap_estimators = find_estimators(
^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/utils/wrangler.py", line 354, in find_estimators
e = fm.FieldmapEstimation(
^^^^^^^^^^^^^^^^^^^^^^
File "<attrs generated init sdcflows.fieldmaps.FieldmapEstimation>", line 7, in __init__
self.__attrs_post_init__()
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/fieldmaps.py", line 434, in __attrs_post_init__
_estimators[self.bids_id] = self.paths()
~~~~~~~~~~~^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/utils/bimap.py", line 144, in __setitem__
raise KeyError(
KeyError: "'phdiff_fmap0' is already in mapping"
Additional information / screenshots
Even when the identifier is the same across all subjects, it works normal if I run fmriprep for each subject separately via --participant-label. If we need to use different identifiers for each subjects, then the behavior of fmriprep differs when run for one subject versus run for multiple subjects, which seems unnatural.
Last but not least, if the identifier must be unique, why bids-validator passed with no error?
Thanks for reporting this! I never actually run multiple subjects at once, so didn't catch this. This will be an issue with sdcflows, so I'm going to transfer the issue there. Without any deep investigation, I suspect this is going to be a fairly high-effort fix. On your end, I would suggest renaming your B0Fields and moving on.
Hi all,
I think I am running into the same / similar error using multi-echo fMRI sequences. It is possible, that the error is on my side and what I am trying to do is not an intended use.
I have some multi-echo fMRI acquired with the CMRR sequence in AP direction and checked "inversed polarity" (IIRC) to acquire some volumes using the same sequence in PA direction. And I am now trying to only estimate the distortion map on the first echo and applying it to the other two.
So I have
func/sub-01_ses-01_task-rest_dir-AP_run-01_echo-1_part-mag_bold.nii.gzfunc/sub-01_ses-01_task-rest_dir-AP_run-01_echo-2_part-mag_bold.nii.gzfunc/sub-01_ses-01_task-rest_dir-AP_run-01_echo-3_part-mag_bold.nii.gz
And renamed (for BIDS compliance):
fmap/sub-01_ses-01_dir-PA_run-01_echo-1_epi.nii.gztofmap/sub-01_ses-01_dir-PA_run-01_epi.nii.gz
I am now trying to create only one distortion map by adding:
- "B0FieldIdentifier": "pepolar_ses-01_fmap4"
- "IntendedFor": "ses-01/func/sub-01_ses-01_task-rest_dir-AP_run-01_echo-1_part-mag_bold.nii.gz"
to
fmap/ses-01_dir-PA_run-01_epi.json
And adding
- "B0FieldIdentifier": "pepolar_ses-01_fmap4"
to all three echos (
sub-01_ses-01_task-rest_dir-AP_run-01_echo-[1-3]_part-mag_bold.json)
Which basically leads to the same error:
Process Process-2:
Traceback (most recent call last):
File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/conda/envs/fmriprep/lib/python3.11/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/cli/workflow.py", line 126, in build_workflow
retval['workflow'] = init_fmriprep_wf()
^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 94, in init_fmriprep_wf
single_subject_wf = init_single_subject_wf(subject_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 515, in init_single_subject_wf
fmap_estimators, estimator_map = map_fieldmap_estimation(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/fmriprep/workflows/base.py", line 763, in map_fieldmap_estimation
fmap_estimators = find_estimators(
^^^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/utils/wrangler.py", line 483, in find_estimators
e = fm.FieldmapEstimation(
^^^^^^^^^^^^^^^^^^^^^^
File "<attrs generated init sdcflows.fieldmaps.FieldmapEstimation>", line 7, in __init__
self.__attrs_post_init__()
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/fieldmaps.py", line 434, in __attrs_post_init__
_estimators[self.bids_id] = self.paths()
~~~~~~~~~~~^^^^^^^^^^^^^^
File "/opt/conda/envs/fmriprep/lib/python3.11/site-packages/sdcflows/utils/bimap.py", line 144, in __setitem__
raise KeyError(
KeyError: "'pepolar_ses-01_fmap4' is already in mapping"
Addition: Just using IntendedFor (as I saw in some discussion on Neurostars works though), but it now appears to use the 2nd echo as a field-map i.e., fmap/sub-01_ses-01_dir-PA_run-01_echo-2_epi.nii.gz and not the renamed fmap/sub-01_ses-01_dir-PA_run-01_epi.nii.gz. Basically, it's matching correctly, but I think it's using the 2nd echo (not the one that is defined in the IntendedFor), which I assume is because of alphabetical sorting.
So for now I won't rename my fmaps and keep the echo in there and only use IntendedFor. I would've like to use B0Field identifiers though, to be certain that the correct fmaps are applied.
Thanks for reporting this! I never actually run multiple subjects at once, so didn't catch this. This will be an issue with sdcflows, so I'm going to transfer the issue there. Without any deep investigation, I suspect this is going to be a fairly high-effort fix. On your end, I would suggest renaming your B0Fields and moving on.
B0FieldIdentifier should be unique. That way, you can do things like correcting an EPI image with a fieldmap from another session, or in a dataset where only 30% of the subjects have a fieldmap, correct with an average of those fieldmaps.
This might need to be made clear in BIDS, then. Global uniqueness is not mentioned at all in the definition or surrounding text.