Perl issue in fresurfer

[DimuthuHemachandra]

I have been trying to run recon-all using the freesurfer mini version. Looks like there is an issue with Perl. Freesurfer says that they have fixed it in their dev version. Can you please look into this? Thanks.

[kaczmarj]

thanks @DimuthuHemachandra - can you please include the following:

• neurodocker command used to build the image
• command that produced the perl error
• full stacktrace of the error

[DimuthuHemachandra]

Here is the command and the output I get. When I shell into the docker image recon-all command is available but not the nu_correct. I am new to docker containers and I would appreciate your help. Thanks.

Command that I build the container: docker build -t dimuthu/hcp_360 .

command that produced the perl error: recon-all

error: Dockerfile.pdf

Dimuthus-MacBook-Pro:code dimuthu$ docker run --rm -it -v /Users/dimuthu/Documents/Robarts/project2/hcp_360/GIRAFFE/code/bids:/in -v /Users/dimuthu/Documents/Robarts/project2/hcp_360/GIRAFFE/code/derivatives:/out -v $FREESURFER_HOME/license.txt:/opt/freesurfer-6.0.0-min/license.txt dimuthu/hcp_360 /in /out participant --participant_label 01

200302-18:17:29,836 nipype.utils INFO:

 No new version available.

sub_id =

200302-18:17:30,81 nipype.workflow INFO:

 Workflow MyWorkflow settings: ['check', 'execution', 'logging', 'monitoring']

200302-18:17:30,113 nipype.workflow INFO:

 Running in parallel.

200302-18:17:30,115 nipype.workflow INFO:

 [MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 5.01/5.01, Free processors: 8/8.

200302-18:17:30,185 nipype.workflow INFO:

 [Node] Setting-up "MyWorkflow.SelectFiles" in "/out/work/MyWorkflow/_sub_id_01/SelectFiles".

200302-18:17:30,232 nipype.workflow INFO:

 [Node] Running "SelectFiles" ("nipype.interfaces.io.SelectFiles")

200302-18:17:30,263 nipype.workflow INFO:

 [Node] Finished "MyWorkflow.SelectFiles".

200302-18:17:32,122 nipype.workflow INFO:

 [Job 0] Completed (MyWorkflow.SelectFiles).

200302-18:17:32,125 nipype.workflow INFO:

 [MultiProc] Running 0 tasks, and 1 jobs ready. Free memory (GB): 5.01/5.01, Free processors: 8/8.

200302-18:17:32,202 nipype.workflow INFO:

 [Node] Setting-up "MyWorkflow.freesurfer_recon_all" in "/out/work/MyWorkflow/_sub_id_01/freesurfer_recon_all".

200302-18:17:32,255 nipype.interface INFO:

 resume recon-all : recon-all -all -subjid 01 -sd /out/freesurfer 

200302-18:17:32,259 nipype.workflow INFO:

 [Node] Running "freesurfer_recon_all" ("nipype.interfaces.freesurfer.preprocess.ReconAll"), a CommandLine Interface with command:

recon-all -all -subjid 01 -sd /out/freesurfer

200302-18:17:32,284 nipype.interface INFO:

 resume recon-all : recon-all -all -subjid 01 -sd /out/freesurfer 

200302-18:17:33,115 nipype.workflow WARNING:

 Storing result file without outputs

200302-18:17:33,122 nipype.workflow WARNING:

 [Node] Error on "MyWorkflow.freesurfer_recon_all" (/out/work/MyWorkflow/_sub_id_01/freesurfer_recon_all)

200302-18:17:34,135 nipype.workflow ERROR:

 Node freesurfer_recon_all.a0 failed to run on host 309b1175b1b3.

200302-18:17:34,137 nipype.workflow ERROR:

 Saving crash info to /code/crash-20200302-181734-root-freesurfer_recon_all.a0-0398d63b-abcc-4665-9858-2e41c97ace8e.pklz

Traceback (most recent call last):

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node

result["result"] = node.run(updatehash=updatehash)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run

result = self._run_interface(execute=True)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface

return self._run_command(execute)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command

result = self._interface.run(cwd=outdir)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 397, in run

runtime = self._run_interface(runtime)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 792, in _run_interface

self.raise_exception(runtime)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 723, in raise_exception

).format(**runtime.dictcopy())

RuntimeError: Command:

recon-all -all -subjid 01 -sd /out/freesurfer

Standard output:

Subject Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c

Current Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c

INFO: SUBJECTS_DIR is /out/freesurfer

Actual FREESURFER_HOME /opt/freesurfer-6.0.0-min

-rw-rw-r-- 1 root root 16877 Mar 2 18:15 /out/freesurfer/01/scripts/recon-all.log

Linux 309b1175b1b3 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 GNU/Linux

'/opt/freesurfer-6.0.0-min/bin/recon-all' -> '/out/freesurfer/01/scripts/recon-all.local-copy'

#--------------------------------------------

#@# MotionCor Mon Mar 2 18:17:33 UTC 2020

ERROR: no run data found in /out/freesurfer/01/mri. Make sure to

have a volume called 001.mgz in /out/freesurfer/01/mri/orig.

If you have a second run of data call it 002.mgz, etc.

See also: http://surfer.nmr.mgh.harvard.edu/fswiki/FsTutorial/Conversion

Linux 309b1175b1b3 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 GNU/Linux

recon-all -s 01 exited with ERRORS at Mon Mar 2 18:17:33 UTC 2020

For more details, see the log file /out/freesurfer/01/scripts/recon-all.log

To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting

Standard error:

nu_correct: Command not found.

Return code: 1

200302-18:17:34,141 nipype.workflow INFO:

 [MultiProc] Running 0 tasks, and 0 jobs ready. Free memory (GB): 5.01/5.01, Free processors: 8/8.

200302-18:17:36,133 nipype.workflow INFO:

 ***********************************

200302-18:17:36,134 nipype.workflow ERROR:

 could not run node: MyWorkflow.freesurfer_recon_all.a0

200302-18:17:36,134 nipype.workflow INFO:

 crashfile: /code/crash-20200302-181734-root-freesurfer_recon_all.a0-0398d63b-abcc-4665-9858-2e41c97ace8e.pklz

200302-18:17:36,134 nipype.workflow INFO:

 ***********************************

Traceback (most recent call last):

File "workflow.py", line 174, in

analysisflow.run(plugin=plugin, plugin_args=plugin_args)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 632, in run

runner.run(execgraph, updatehash=updatehash, config=self.config)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/plugins/base.py", line 192, in run

report_nodes_not_run(notrun)

File "/opt/miniconda-latest/lib/python3.7/site-packages/nipype/pipeline/plugins/tools.py", line 98, in report_nodes_not_run

("Workflow did not execute cleanly. " "Check log for details")

RuntimeError: Workflow did not execute cleanly. Check log for details

Dimuthus-MacBook-Pro:code dimuthu$

[DimuthuHemachandra]

/opt/freesurfer-6.0.0-min/mni/lib/perl5/5.8.5/MNI This folder is empty. I think that is what causing this issue.

[axiezai]

I'm running into the same error in singularity image with freesurfer-6.0.0-min, the image was created with neurodocker 0.7.0:

Iteration 1 Mon May 25 13:52:03 PDT 2020
nu_correct -clobber ./tmp.mri_nu_correct.mni.67113/nu0.mnc ./tmp.mri_nu_correct.mni.67113/nu1.mnc -tmpdir ./tmp.mri_nu_correct.mni.67113/0/ -iterations 1000 -distance 50
nu_correct: Command not found.
ERROR: nu_correct

Would gladly provide any additional log files / info to help solve this issue :)

[axiezai]

I think I found the problem... I used the same neurodocker to generate both singularity and docker recipe files.

In the singularity image, echo $PATH returns:

/opt/miniconda-latest/envs/tracts/bin:/opt/miniconda-latest/condabin:/opt/miniconda-latest/bin:/opt/freesurfer-6.0.0-min/bin:/opt/fsl-6.0.1/bin:/opt/ants-2.3.1:/opt/mrtrix3-3.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

whereas in the docker image, echo $PATH returns:

/opt/freesurfer-6.0.0-min/bin:/opt/freesurfer-6.0.0-min/fsfast/bin:/opt/freesurfer-6.0.0-min/tktools:/opt/fsl-6.0.1/bin:/opt/freesurfer-6.0.0-min/mni/bin:/opt/miniconda-latest/bin:/opt/freesurfer-6.0.0-min/bin:/opt/fsl-6.0.1/bin:/opt/ants-2.3.1:/opt/mrtrix3-3.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Basically, the singularity recipe is missing /opt/fsl-6.0.1/bin:/opt/freesurfer-6.0.0-min/mni/bin, where nu_correct is located. Appending this path manually might be a temporary solution, but would be nice if the neurodocker generate command includes this path.

EDIT: After appending the path, now running into the following issue in the singularity image when running recon-all:

Standard error:
Can't locate MNI/Startup.pm in @INC (you may need to install the MNI::Startup module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /opt/freesurfer-6.0.0-min/mni/bin/nu_correct line 37.
BEGIN failed--compilation aborted at /opt/freesurfer-6.0.0-min/mni/bin/nu_correct line 37.

[PeerHerholz]

Ahoi hoi folks,

I'm having the same problem as well, however within a docker container and using neurodocker based on the current main branch. I need to run source /opt/freesurfer-6.0.0/SetUpFreeSurfer.sh within the container to make things work despite I added it to the entrypoint of the container.

[shnizzedy]

After appending the path, now running into the following issue in the singularity image when running recon-all:

Standard error:
Can't locate MNI/Startup.pm in @INC (you may need to install the MNI::Startup module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /opt/freesurfer-6.0.0-min/mni/bin/nu_correct line 37.
BEGIN failed--compilation aborted at /opt/freesurfer-6.0.0-min/mni/bin/nu_correct line 37.

― https://github.com/ReproNim/neurodocker/issues/321#issuecomment-635521728

The various environment variables that are $FREESURFER_HOME/** are using /usr/local/freesurfer as $FREESURFER_HOME:

FREESURFER_HOME   /opt/freesurfer-6.0.0-min
FSFAST_HOME       /usr/local/freesurfer/fsfast
SUBJECTS_DIR      /usr/local/freesurfer/subjects
MNI_DIR           /usr/local/freesurfer/mni
MINC_BIN_DIR      /usr/local/freesurfer/mni/bin
MINC_LIB_DIR      /usr/local/freesurfer/mni/lib
PERL5LIB          /usr/local/freesurfer/mni/share/perl5

As a workaround, one could set each of those to the correct in-image path, e.g.

ENV FREESURFER_HOME="/opt/freesurfer-6.0.0-min" \
    PERL5LIB="$FREESURFER_HOME/mni/share/perl5" \
    FSFAST_HOME="$FREESURFER_HOME/fsfast" \
    SUBJECTS_DIR="$FREESURFER_HOME/subjects" \
    MNI_DIR="$FREESURFER_HOME/mni" \
    MINC_BIN_DIR="$MNI_DIR/bin" \
    MINC_LIB_DIR="$MNI_DIR/lib" \
    PATH="$PATH:$MINC_BIN_DIR"

@axiezai, correcting $PERL5LIB resolves Can't locate MNI/Startup.pm in @INC

[shnizzedy]

After a bit more digging, for me the surprising environment variables (e.g., the ones starting with /usr/local/freesurfer) were coming from my host environment. Starting a container with --cleanenv (singularity run --cleanenv ) avoids the issue entirely.

[kaczmarj]

@PeerHerholz - can you include your neurodocker command and the command you are using to run the docker image?

@DimuthuHemachandra and @axiezai - does the solution (https://github.com/ReproNim/neurodocker/issues/321#issuecomment-811924111) solve your issue?

[axiezai]

i can confirm that using singularity -e (flag for a clean environnment upon container start) has been my default singularity usage lately (since 3.6 update). I had also removed freesurfer from my containers so not sure if it's helping with this issue in particular.

But from what I remember, I had to manually append a few things to my container PATH for freesurfer and miniconda, for Example: export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/opt/miniconda-latest/lib/" was required but not part of the recipe.

Not sure if neurodocker is simply missing these environment variables or singularity/docker is the problem.

[shnizzedy]

Not sure if this helps with

But from what I remember, I had to manually append a few things to my container PATH for freesurfer and miniconda, for Example: export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu:/opt/miniconda-latest/lib/" was required but not part of the recipe. ― https://github.com/ReproNim/neurodocker/issues/321#issuecomment-812635196

but number 7 in Singularity's Support for Docker and OCI: Best Practices is

Ensure calls to ldconfig are executed towards the end of build specifications (e.g., Dockerfile), so that the library cache is updated when the container is created.

As far as I know, neurodocker doesn't do that automatically (except for in the AFNI installation), so I just include --run 'ldconfig' as the final flag when I run neurodocker

[kaczmarj]

@shnizzedy - thanks for pointing this out. can you please submit a new issue about running ldconfig? we need to fix that, and we can track progress in that issue

@axiezai - very possible that neurodocker is the problem. i will need more time to look into the problem

[PeerHerholz]

Ahoi hoi folks,

sorry for the late reply. In my case, there were a lot of cross-dependencies, etc. wrt LD_LIBRARY_PATH et al., as well as exclude pattern issues.... IIRC, I updated these things and added --add-to-entrypoint "source /opt/freesurfer-6.0.0/SetUpFreeSurfer.sh" \ which resolved the issue on my end.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.