snakemake-wrappers
snakemake-wrappers copied to clipboard
Strelk2 bug: unsupported pickle protocol:3
Snakemake version
5.24.2
Describe the bug
I'm getting an error when using the Strelka2
wrapper when using the --use-singularity
and --use-conda
commands in conjunction but not when --use-conda
is used alone.
I'm wondering if this is problem with the wrapper or the singularity image? If it's the image can you suggest a better one to use please
Logs
Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 strelka2
1
[Thu Sep 24 17:16:03 2020]
Job 0: --- Call germline variants with Strelka2.
python /home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py
Activating singularity image /home/moldach/wrappers/SUBSET/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f
Traceback (most recent call last):
File "/home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py", line 3, in <module>
x00sampleqhK\x00N\x86qish\x15]qj(h\x17h\x18eh\x17h\x19h\x1e\x85qkRql(h\x1e)}qmh"h\x17sNtqnbh\x18h\x19h\x1e\x85qoRqp(h\x1e)}qqh"h\x18sNtqrbX\x06\x00\x00\x00sampleqsheubX\x07\x00\x00\x00threadsqtK\x08$
File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 1388, in loads
return Unpickler(file).load()
File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 892, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3
[Thu Sep 24 17:16:18 2020]
Error in rule strelka2:
jobid: 0
output: strelka/MTG324
log: logs/bowtie2/MTG324.log (check log file(s) for error message)
conda-env: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f
RuleException:
CalledProcessError in line 453 of /home/moldach/wrappers/SUBSET/Snakefile:
Command ' singularity exec --home /home/moldach/wrappers/SUBSET --bind /home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages:/mnt/snakemake /home/moldach/wrappers/SUBSET/.snakemake/singula$
File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2189, in run_wrapper
File "/home/moldach/wrappers/SUBSET/Snakefile", line 453, in __rule_strelka2
File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 529, in _callback
File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/concurrent/futures/thread.py", line 57, in run
File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 515, in cached_or_run
File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2201, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Minimal example
Singularity container
singularity: "docker://continuumio/miniconda3:4.5.11"
Rule
def getDeduppedBamsIndex(sample):
return(list(os.path.join(aligns_dict[sample],"{0}.sorted.dedupped.bam.bai".format(sample,pair)) for pair in ['']))
if (config["CALLING_TOOL"]=="strelka2"):
rule strelka2:
input:
fasta=os.path.join(dirs_dict["REF_DIR"],config["REF_GENOME"]),
bam=lambda wildcards: getDeduppedBams(wildcards.sample),
bam_index=lambda wildcards: getDeduppedBamsIndex(wildcards.sample),
fasta_index=os.path.join(dirs_dict["REF_DIR"],GENOME_INDEX)
output:
temp(directory("strelka/{sample}"))
log: os.path.join(dirs_dict["LOG_DIR"],config["ALIGN_TOOL"],"{sample}.log")
message: """--- Call germline variants with Strelka2."""
threads: 8
resources:
mem=4000,
time=100
params:
config_extra="",
run_extra=""
wrapper:
"0.65.0/bio/strelka/germline"
"""
Re: the error
unsupported pickle protocol: 3
"Perhaps the person who created the files wasn’t aware of the fact that the default protocol for pickle in Python 3 is “3”, but that this is backward-incompatible. "
from this post: https://stackoverflow.com/questions/34571063/i-have-pickled-files-using-protocol-3-in-python3-and-now-i-need-to-unpickle-the/34571081#34571081