varCA icon indicating copy to clipboard operation
varCA copied to clipboard

ldpaths error when using prepare.yaml conda env

Open aryarm opened this issue 3 years ago • 1 comments

When running the prepare subworkflow, users of VarCA might occasionally run into the following error:

$CONDA_PREFIX/lib/R/bin/R: line 238: $CONDA_PREFIX/lib/R/etc/ldpaths: No such file or directory

This error can happen when Snakemake activates the prepare conda environment from two different processes at the same time on an NFS file system. This is usually a rare occurrence unless you are running many samples through the pipeline simultaneously. The problem (and some potential solutions) is discussed at length in this Github issue.

If this error happens to you, it is essentially because of bad luck. Because the issue leads to a corrupted R installation within the prepare conda environment that Snakemake created, we recommend deleting the prepare conda environment, so that Snakemake can automatically recreate it afresh. You can determine where the prepare conda environment was created by running snakemake --list-conda-envs and then delete it using rm -r. To reduce your chances of running into this problem in the future, you can limit the number of jobs that run simultaneously by passing a smaller number of cores to Snakemake via the -j parameter in the run.bash script. The tradeoff of this approach is that VarCA may take longer to finish executing.

There are currently no plans for us to resolve this issue, since it is a bug within conda-forge and not within VarCA. But feel free to let us know below if you have any suggestions.

aryarm avatar Oct 23 '20 04:10 aryarm