An error

[Edison2021]

Hi there

I ran a trial test using the command line as: princess all --directory analysis --ReadType ccs --ref /home/bwu3/Used_V41_P13_107/p107.ens.fasta --jobs 7 --sampleFiles HiFi.fastq.gz --latency-wait 200 -p -c 2 --verbose

An error popped up below ######## Error in rule mergeAlign: jobid: 4 output: /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/data.bam log: /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/merge.log (check log file(s) for error message) conda-env: /mnt/chsrhome/bwu3/test_file/analysis/.snakemake/conda/a34f2739af6ede4165125922d923477d_ shell:

    samtools merge -@ 5 /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/data.bam /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/HiFi.fastq.gz.bam > /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/merge.log 2>&1

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
cluster_jobid: 3999

Error executing rule mergeAlign on cluster (jobid: 4, external: 3999, jobscript: /mnt/chsrhome/bwu3/test_file/analysis/.snakemake/tmp.a0pkihz4/snakejob.mergeAlign.4.sh). For error details see the cluster log and the log files of the involved rule(s). ########

Could you please give me some suggestions? Thanks

[MeHelmy]

Hi,

Can you please share the error log available here: /mnt/chsrhome/bwu3/test_file/analysis/align/minimap/merge.log

Thanks, Medhat

[Edison2021]

The merge.log is empty.

[MeHelmy]

Hmm, do you see a file contains the 3999 prefix from the job cluster_jobid? It could be in the directory called analysis or in a subdirectory called snake_log

[Edison2021]

In the file snakejob.mergeAlign.e3999, the message is bash: xmalloc: .././locale.c:81: cannot allocate 2 bytes (0 bytes allocated)

[MeHelmy]

So, I gave the merging these parameters:

  time: "24:00:00"
  nCPUs: "8"
  mem: 50G

As you can see here https://github.com/MeHelmy/princess/blob/master/cluster/cluster_config.yaml

Princess if the previous configuration is not found it will just adapt what you have. But, from the error, it appears that you do not have enough memory.

How much memory CPU available on your node? (You can check also with the IT administrator.)

Best, Medhat

[EDISON2022W]

Hi Medhat I disabled the cluster using -e, and it worked for the alignment, merging, and calling SV steps but failed in the calling SNV step. ############### In the install.sh wget https://github.com/squeaky-pl/portable-pypy/releases/download/pypy3.6-7.2.0/pypy3.6-7.2.0-linux_x86_64-portable.tar.bz2 mv pypy* pypy.tar.bz2 tar -xvjf pypy.tar.bz2 mv pypy3.* pypy ############### In the config.yaml clair_pypy: "/home/source/Clair/pypy3/pypy3.5-7.0.0-linux_x86_64-portable/bin/pypy"

Does the different pypy name cause the calling SNV problem?

Best Edison

[MeHelmy]

Hi, in the installation script. I install the required pypy and export it. Like here:

cd $clair_dir
wget https://github.com/squeaky-pl/portable-pypy/releases/download/pypy3.6-7.2.0/pypy3.6-7.2.0-linux_x86_64-portable.tar.bz2
mv pypy* pypy.tar.bz2
tar -xvjf pypy.tar.bz2
mv pypy3.* pypy
# tar -xvjf pypy3.6-7.2.0-linux_x86_64-portable.tar.bz2
cd $work_dir
export PATH=$clair_dir/pypy/bin:$PATH


wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py
pypy3 -m ensurepip
#pypy3 -m pip install --no-cache-dir intervaltree blosc
pypy3 -m pip install --no-cache-dir intervaltree
pypy3 -m pip install blosc==1.8.3

The clair_pypy: "/home/source/Clair/pypy3/pypy3.5-7.0.0-linux_x86_64-portable/bin/pypy" is absolute, I do not use it anymore

Can you share the error you get?

[EDISON2022W]

The error is

Traceback (most recent call last): File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486_/bin/clair/../clair.py", line 92, in main() File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486_/bin/clair/../clair.py", line 79, in main submodule = import_module("%s.%s" % (directory, submodule_name)) File "/mnt/chsrhome/bbww/test_file/analysis/bin/pypy/lib/pypy3.9/importlib/init.py", line 127, in import_module return bootstrap.gcd_import(name[level:], package, level) File "", line 1030, in gcd_import File "", line 1007, in find_and_load File "", line 986, in find_and_load_unlocked File "", line 680, in load_unlocked File "/frozen importlib.bootstrap_external", line 863, in exec_module File "", line 228, in call_with_frames_removed File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/clair/call_var.py", line 5, in import numpy as np ModuleNotFoundError: No module named 'numpy' [faidx] Truncated sequence: 2:210919337-243193529 [faidx] Truncated sequence: 2:210919337-243193529 Traceback (most recent call last): File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/clair/../clair.py", line 92, in main() File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/clair/../clair.py", line 86, in main submodule.main() File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/dataPrepScripts/CreateTensor.py", line 440, in main OutputAlnTensor(args) File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/dataPrepScripts/CreateTensor.py", line 372, in OutputAlnTensor tensor_fp.stdin.write(l) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/clair/../clair.py", line 92, in main() File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/clair/../clair.py", line 86, in main submodule.main() File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486/bin/dataPrepScripts/ExtractVariantCandidates.py", line 450, in main make_candidates(args) File "/mnt/chsrhome/bbww/test_file/analysis/.snakemake/conda/680978645fbe5f728b69ce9ba0c7f486_/bin/dataPrepScripts/ExtractVariantCandidates.py", line 367, in make_candidates can_fp.stdin.write(output) BrokenPipeError: [Errno 32] Broken pipe

By the way, I have installed NumPy

Python 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.

import numpy numpy.version.version '1.21.5'