snakemake-wrappers icon indicating copy to clipboard operation
snakemake-wrappers copied to clipboard

Strelk2 bug: unsupported pickle protocol:3

Open moldach opened this issue 3 years ago • 1 comments

Snakemake version 5.24.2

Describe the bug

I'm getting an error when using the Strelka2 wrapper when using the --use-singularity and --use-conda commands in conjunction but not when --use-conda is used alone.

I'm wondering if this is problem with the wrapper or the singularity image? If it's the image can you suggest a better one to use please

Logs

Workflow defines that rule get_vep_cache is eligible for caching between workflows (use the --cache argument to enable this).
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 8
Rules claiming more threads will be scaled down.
Job counts:
	count   jobs
        1	strelka2
        1

[Thu Sep 24 17:16:03 2020]
Job 0: --- Call germline variants with Strelka2.

python /home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py
Activating singularity image /home/moldach/wrappers/SUBSET/.snakemake/singularity/d7617773b315c3abcb29e0484085ed06.simg
Activating conda environment: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f
Traceback (most recent call last):
  File "/home/moldach/wrappers/SUBSET/.snakemake/scripts/tmpoc6_0bnr.wrapper.py", line 3, in <module>
    x00sampleqhK\x00N\x86qish\x15]qj(h\x17h\x18eh\x17h\x19h\x1e\x85qkRql(h\x1e)}qmh"h\x17sNtqnbh\x18h\x19h\x1e\x85qoRqp(h\x1e)}qqh"h\x18sNtqrbX\x06\x00\x00\x00sampleqsheubX\x07\x00\x00\x00threadsqtK\x08$
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 1388, in loads
    return Unpickler(file).load()
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
  File "/home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f/lib/python2.7/pickle.py", line 892, in load_proto
    raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3
[Thu Sep 24 17:16:18 2020]
Error in rule strelka2:
    jobid: 0
    output: strelka/MTG324
    log: logs/bowtie2/MTG324.log (check log file(s) for error message)
    conda-env: /home/moldach/wrappers/SUBSET/.snakemake/conda/1995398f

RuleException:
CalledProcessError in line 453 of /home/moldach/wrappers/SUBSET/Snakefile:
Command ' singularity exec --home /home/moldach/wrappers/SUBSET  --bind /home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages:/mnt/snakemake /home/moldach/wrappers/SUBSET/.snakemake/singula$
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2189, in run_wrapper
  File "/home/moldach/wrappers/SUBSET/Snakefile", line 453, in __rule_strelka2
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 529, in _callback
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/concurrent/futures/thread.py", line 57, in run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 515, in cached_or_run
  File "/home/moldach/anaconda3/envs/snakemake/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2201, in run_wrapper
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Minimal example

Singularity container

singularity: "docker://continuumio/miniconda3:4.5.11"

Rule

def getDeduppedBamsIndex(sample):
  return(list(os.path.join(aligns_dict[sample],"{0}.sorted.dedupped.bam.bai".format(sample,pair)) for pair in ['']))

if (config["CALLING_TOOL"]=="strelka2"):
        rule strelka2:
            input:
                    fasta=os.path.join(dirs_dict["REF_DIR"],config["REF_GENOME"]),
                    bam=lambda wildcards: getDeduppedBams(wildcards.sample),
                    bam_index=lambda wildcards: getDeduppedBamsIndex(wildcards.sample),
                    fasta_index=os.path.join(dirs_dict["REF_DIR"],GENOME_INDEX)
            output:
                    temp(directory("strelka/{sample}"))
            log: os.path.join(dirs_dict["LOG_DIR"],config["ALIGN_TOOL"],"{sample}.log")
            message: """--- Call germline variants with Strelka2."""
            threads: 8
            resources:
                    mem=4000,
                    time=100
            params:
                    config_extra="",
                    run_extra=""
            wrapper:
                    "0.65.0/bio/strelka/germline"
            """


moldach avatar Sep 24 '20 23:09 moldach

Re: the error

unsupported pickle protocol: 3

"Perhaps the person who created the files wasn’t aware of the fact that the default protocol for pickle in Python 3 is “3”, but that this is backward-incompatible. "

from this post: https://stackoverflow.com/questions/34571063/i-have-pickled-files-using-protocol-3-in-python3-and-now-i-need-to-unpickle-the/34571081#34571081

moldach avatar Sep 24 '20 23:09 moldach