Fails to find DWI pairs

[aghayoor]

Hi developer team,

I am trying to run QSIPrep on some PPMI data. In their acquisition set up, they have acquired two sessions of DWI data in opposite phase encoding direction, that can be used for susceptibility artifact correction. For this, I used --distortion-group-merge average argument in qsiprep.

Here is the full command line I used to run qsiprep:

qsiprep-docker $datadir/BIDS/ $outdir participant --participant-label 01 --output-resolution 1.2 --fs-license-file ./fs_license.txt --distortion-group-merge average --verbose

However, I receive this error:

Node: qsiprep_wf.single_subject_01_wf.sub_01_final_merge_wf.distortion_merger Working directory: /tmp/work/qsiprep_wf/single_subject_01_wf/sub_01_final_merge_wf/distortion_merger

Node inputs:

b0_refs = b0_threshold = 100 bids_dwi_files = bval_files = bvec_files = carpetplot_data = denoising_confounds = dwi_files = harmonize_b0_intensities = True original_bvec_files = raw_concatenated_files = verbose = False

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node result["result"] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run result = self._run_interface(execute=True) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface return self._run_command(execute) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command result = self._interface.run(cwd=outdir) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 428, in run runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 168, in _run_interface image_pairs, averaged_raw_bvec = find_image_pairs(original_bvecs, bvals, assignments) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 241, in find_image_pairs return pairs, bvecs UnboundLocalError: local variable 'pairs' referenced before assignment

I appreciate your feedback and help here.

Thank you, Ali

[mattcieslak]

Hi! Can you confirm that both images have the same exact set of bvals and bvecs?

[aghayoor]

Hi, bvals are exactly the same, but bvecs are in opposite directions. Also, bvecs are different from the 3rd number after the decimal point.

[aghayoor]

Here is bvecs for run1:

0 -0.203737 -0.198274 -0.402136 -0.404018 -0.202985 -0.854935 -0.730935 -0.406286 -0.73048 -0.652278 -0.323539 -0.324763 -0.650725 -0.979138 -0.854525 -0.00141114 -0.00315668 -0. 653786 -0.329346 -0.198481 -0.203793 -0.652591 -0.325359 -0.201376 -0.404296 -0.731671 -0.729202 -0.854296 -0.856057 -0.979588 0 0 0 0.513951 -0.516044 -0.171602 0.726775 0.93967 0.514633 0.515185 0.171277 0.17193 0.727757 0.9402 0.515615 0.51588 0.171844 0.17206 -0.726605 -0.939524 -0.514701 -0.514842 0. 172601 -0.171561 -0.727195 -0.939829 -0.939191 -0.728164 -0.172033 -0.515224 -0.51533 -0.17168 -0.171357 0 0 0 -0.833274 0.833298 0.899355 -0.555489 -0.275351 -0.0651115 -0.447569 -0.89755 -0.660938 0.211904 0.106522 0.792887 0.557158 0.108439 0.490084 -0.687054 -0.342468 -0.55466 -0. 791498 0.964787 -0.963865 -0.212868 -0.104228 0.278151 0.553463 0.659593 0.450343 0.0679233 -0.487537 -0.105092 0 0

And run 2:

0 0.203855 0.198183 0.400997 0.404296 0.203078 0.855078 0.731993 0.407049 0.732011 0.651908 0.323426 0.324103 0.649915 0.978835 0.853087 0.00141654 0.00316412 0.654976 0.330028 0. 198262 0.204114 0.653159 0.325469 0.201027 0.404018 0.73014 0.728361 0.854172 0.857263 0.979874 0 0 0 -0.515519 0.514474 0.17163 -0.728164 -0.940545 -0.514772 -0.515983 -0.172358 -0.171941 -0.727293 -0.939874 -0.514101 -0.514911 -0.171734 -0.171964 0.728056 0.940637 0.515401 0. 516354 -0.171573 0.171678 0.727666 0.940138 0.938405 0.726775 0.172021 0.514444 0.515241 0.17163 0.17146 0 0 0 0.832276 -0.83429 -0.899858 0.553463 0.272275 0.0620566 0.444913 0.896997 0.659239 -0.214618 -0.109697 -0.794139 -0.558996 -0.111306 -0.492616 0.685517 0.339399 0.552601 0. 790228 -0.965016 0.963776 0.209489 0.101049 -0.281043 -0.555489 -0.66129 -0.45259 -0.070126 0.485432 0.102216 0 0

[mattcieslak]

That's very interesting. In theory you should be able to overwrite the bvecs from the second run with those from the first. Another option would be to do --distortion-merge concat and you can manually merge them. I personally have always preferred the concat option because it gives you more data for modelling.

[aghayoor]

Thanks @mattcieslak ,

I used --distortion-group-merge concat, and I got a different error message:

Node: qsiprep_wf.single_subject_01_wf.sub_01_final_merge_wf.distortion_merger Working directory: /tmp/work/qsiprep_wf/single_subject_01_wf/sub_01_final_merge_wf/distortion_merger

Node inputs:

b0_refs = b0_threshold = 100 bids_dwi_files = bval_files = bvec_files = carpetplot_data = denoising_confounds = dwi_files = harmonize_b0_intensities = True raw_concatenated_files =

Traceback (most recent call last): File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node result["result"] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run result = self._run_interface(execute=True) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface return self._run_command(execute) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command result = self._interface.run(cwd=outdir) File "/usr/local/miniconda/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 428, in run runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 64, in _run_interface self.inputs.harmonize_b0_intensities) File "/usr/local/miniconda/lib/python3.7/site-packages/qsiprep/interfaces/dwi_merge.py", line 487, in harmonize_b0s b0_mean = index_img(dwi_nii, b0_indices).get_fdata().mean() File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/image/image.py", line 672, in index_img imgs = check_niimg_4d(imgs) File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/_utils/niimg_conversions.py", line 380, in check_niimg_4d dtype=dtype) File "/usr/local/miniconda/lib/python3.7/site-packages/nilearn/_utils/niimg_conversions.py", line 296, in check_niimg raise DimensionError(len(niimg.shape), ensure_ndim) nilearn._utils.exceptions.DimensionError: Input data has incompatible dimensionality: Expected dimension is 4D and you provided a 3D image. See http://nilearn.github.io/manipulating_images/input_output.html.

I checked both DWI acquisitions and those are correctly converted to BIDS and are 4D image. Do you have an idea what has caused this error?

Thanks, Ali

[cookpa]

I'm also getting the error

nilearn._utils.exceptions.DimensionError: Input data has incompatible dimensionality: Expected dimension is 4D and you provided a 3D image.

on qsiprep 0.14.2 with HCP data and --distortion-group-merge concat. Trying it with average right now.

[aghayoor]

@cookpa , please update us on the results with average. I ran this option on PPMI data, and reported the error above (It couldn't find the DWI pairs).

[aghayoor]

Hi @mattcieslak , I wanted to follow up here to see if you have any idea about the above error when concat is used? Thank you!

[mattcieslak]

~~Could you try without using any --distortion-group-merge options? It looks like there is a bug in concat~~

I see where this is happening. may be a quick fix

[cookpa]

@aghayoor average worked fine

[aghayoor]

Hi @mattcieslak ,

I ran qsiprep without using any --distortion-group-merge options. It ran with no crash and successfully wrote the preprocessed dwi image to the disk.

However, there are some issues/questions:

• Although, I didn't use any --distortion-group-merge options, the output preprocessed dwi is susceptibility artifact corrected and concatenated! By looking at the screen log, I can confirm that "raw_rpe_concat" and "rpe_concat" nodes and "hmc_sdc_wf" workflow are run. Is susceptibility distortion correction configured by default?

• In the output QC html file, I only see methods and references. Although no error is reported, I don't see any preprocessed QC steps. Any idea why?

Thanks!

[aghayoor]

@mattcieslak ,

In QSIPrep documents, it is mentioned that:

Otherwise (default) the DWI scans will be merged (i.e. their images will be concatenated). The merging affects the pipeline at different stages. If all DWIs in a session are in the same PE direction, they will be merged into a single series. If there are two PE directions detected in the DWI scans and 'fieldmaps' is not in ignore, images are combined according to their PE direction, and their b0 reference images are used to perform SDC. Further complicating this is the FSL workflow, which combines distortion correction with eddy/motion correction and will merge scans with different PE directions.

This suggests that --distortion-group-merge concat is the default behavior, and the average option is added to produce outputs that is directly comparable to the HCP dMRI pipeline. Is that correct?

If so, why using --distortion-group-merge concat in the command-line cause a crash while not using any --distortion-group-merge options work fine?

[mattcieslak]

If so, why using --distortion-group-merge concat in the command-line cause a crash while not using any --distortion-group-merge options work fine?

The concat option is only necessary when you don't use Eddy for motion correction. Eddy internally concats all the images, whereas SHORELine keeps distortion groups separate and would need to concatenate them after motion correction. This is still a bug though

[aghayoor]

@mattcieslak , Got it! Thanks for your comment. It's worthwhile to add this to the documents.

[ajschadler12]

I am having a similar issue. That is, when processing HCP-style scans (two identical multi-shell scans, reverse-PE), if the --distortion-group-merge average and --work-dir arguments are specified, the node qsiprep_wf.single_subject_EPK246_wf.sub_EPK246_acq_multishell_final_merge_wf.distortion_merger crashes, due to UnboundLocalError: local variable 'pairs' referenced before assignment.

I will try and run the data without --work-dir for now. If it indeed crashes due to the same error, I will try running with --distortion-group-merge-concat instead