RedDog icon indicating copy to clipboard operation
RedDog copied to clipboard

Published 20 hours ago • verified publisher icon katholt

Pipeline not progressing past checkBam

Open RMann24 opened this issue 9 years ago • 8 comments

Hi there, I'm having problems getting RedDog to progress any further than Task=RedDog.checkBam. All tasks up to this point are classified as "up-to-date". I have run ‘errorcheck.txt’ but it did not identify any issues. There are "alignBowtiePE.Success" files in the /temp/success/ folder but no "checkBam.pbs" scripts in the /log/ folder. Any help would be appreciated.

Rachel

RMann24 avatar Dec 22 '15 02:12 RMann24

Hi Rachel,

Can you check if the bams are actually being created? Check in the output/temp subfolders - unfortunately, the way the samtools part of the command works is to produce the bam file then populate it. So the pipeline is tricked into thinking the stage has successfully completed, even when it hasn't. Hence checkBam was added to the pipe - this simply checks that the bam file is at least a reasonable size.

BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces a series of tmp files to launch each job - are these being produced (in the reddog folder itself)?

Can you also try run the pipe again so I can see the error? To capture the run just do: rubra RedDog --config your_config --style run > run.txt

when you hit return, then hit 'y' and return and the pipe will run sending the output to the run.txt file.

I'm working today and tomorrow before taking a break until early January, so hopefully we can fix this asap... cheers, David

d-j-e avatar Dec 22 '15 04:12 d-j-e

OOPS - didn't mean to close it!!!

d-j-e avatar Dec 22 '15 04:12 d-j-e

Thanks very much for your help David. The >run.txt only recorded the initial info so I've just cut and past the info from the terminal into the file. I should have mentioned that I am working at AgriBio and RedDog is installed on our server here.

Rachel

Dr Rachel Mann | Research Scientist | Microbiology | Biosciences Research Division Department of Economic Development, Jobs, Transport and Resources AgriBio, Centre for AgriBioscience, 5 Ring Road, Bundoora VIC 3086
T: +61 3 9032 7280 | E: [email protected] W: Wednesday, Thursday and Friday

From: d-j-e [email protected] To: katholt/RedDog [email protected], Cc: RMann24 [email protected] Date: 22/12/2015 03:07 PM Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Hi Rachel, Can you check if the bams are actually being created? Check in the output/temp subfolders - unfortunately, the way the samtools part of the command works is to produce the bam file then populate it. So the pipeline is tricked into thinking the stage has successfully completed, even when it hasn't. Hence checkBam was added to the pipe - this simply checks that the bam file is at least a reasonable size. BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces a series of tmp files to launch each job - are these being produced (in the reddog folder itself)? Can you also try run the pipe again so I can see the error? To capture the run just do: rubra RedDog --config your_config --style run > run.txt when you hit return, then hit 'y' and return and the pipe will run sending the output to the run.txt file. I'm working today and tomorrow before taking a break until early January, so hopefully we can fix this asap... cheers, David — Reply to this email directly or view it on GitHub.

Department of Economic Development, Jobs, Transport and Resources, Government of Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential information. If you are not the intended recipient, you may not distribute or reproduce this e-mail or the attachments. If you have received this message in error, please notify us by return email.

RedDog V1beta.5 - phylogeny run

Copyright (c) 2015, David Edwards, Bernie Pope, Kat Holt All rights reserved. (see README.txt for more details)

Mapping: Bowtie2 V2.2.3 Preset Option: --sensitive-local 3 replicon(s) in GenBank reference GCA_000022865.1_ASM2286v1_genomic 3 replicon(s) to be reported 78 sequence pair(s) to be mapped

Output folder: /group/pathogens/Ba/Ba-redog/

Start Pipeline? (y/n) Starting pipeline... 2291 jobs to be executed in total 2209 jobs left to execute

[rp50@dev3 RedDog]$ rubra RedDog --config RedDog_config_ba --style run > run.txt y Traceback (most recent call last): File "/usr/local/bin/rubra", line 9, in load_entry_point('Rubra==0.1.5', 'console_scripts', 'rubra')() File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/rubra.py", line 66, in main gnu_make_maximal_rebuild_mode=rebuildMode) File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 2680, in pipeline_run raise errt ruffus.ruffus_exceptions.RethrownJobError:

Exceptions running jobs for

'def RedDog.checkBam(...):'

Original exceptions:

Exception #1
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC023.bam, ARC023.alignBowtiePE.Success] -> [ARC023.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #2
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[A0174.bam, A0174.alignBowtiePE.Success] -> [A0174.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #3
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC025.bam, ARC025.alignBowtiePE.Success] -> [ARC025.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #4
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC009.bam, ARC009.alignBowtiePE.Success] -> [ARC009.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #5
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC013.bam, ARC013.alignBowtiePE.Success] -> [ARC013.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #6
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[4599.bam, 4599.alignBowtiePE.Success] -> [4599.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #7
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC079.bam, ARC079.alignBowtiePE.Success] -> [ARC079.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #8
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC096.bam, ARC096.alignBowtiePE.Success] -> [ARC096.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #9
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC083.bam, ARC083.alignBowtiePE.Success] -> [ARC083.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #10
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC027.bam, ARC027.alignBowtiePE.Success] -> [ARC027.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #11
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC095.bam, ARC095.alignBowtiePE.Success] -> [ARC095.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #12
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC137.bam, ARC137.alignBowtiePE.Success] -> [ARC137.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #13
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC127.bam, ARC127.alignBowtiePE.Success] -> [ARC127.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #14
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC061.bam, ARC061.alignBowtiePE.Success] -> [ARC061.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #15
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC106.bam, ARC106.alignBowtiePE.Success] -> [ARC106.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #16
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC028.bam, ARC028.alignBowtiePE.Success] -> [ARC028.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #17
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC086.bam, ARC086.alignBowtiePE.Success] -> [ARC086.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #18
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC120.bam, ARC120.alignBowtiePE.Success] -> [ARC120.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #19
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC026.bam, ARC026.alignBowtiePE.Success] -> [ARC026.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #20
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC112.bam, ARC112.alignBowtiePE.Success] -> [ARC112.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #21
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC073.bam, ARC073.alignBowtiePE.Success] -> [ARC073.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #22
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC108.bam, ARC108.alignBowtiePE.Success] -> [ARC108.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #23
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC049.bam, ARC049.alignBowtiePE.Success] -> [ARC049.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #24
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC094.bam, ARC094.alignBowtiePE.Success] -> [ARC094.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #25
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC021.bam, ARC021.alignBowtiePE.Success] -> [ARC021.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #26
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC075.bam, ARC075.alignBowtiePE.Success] -> [ARC075.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #27
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC077.bam, ARC077.alignBowtiePE.Success] -> [ARC077.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #28
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC115.bam, ARC115.alignBowtiePE.Success] -> [ARC115.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #29
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC135.bam, ARC135.alignBowtiePE.Success] -> [ARC135.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #30
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC125.bam, ARC125.alignBowtiePE.Success] -> [ARC125.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #31
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC062.bam, ARC062.alignBowtiePE.Success] -> [ARC062.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #32
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC011.bam, ARC011.alignBowtiePE.Success] -> [ARC011.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #33
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC063.bam, ARC063.alignBowtiePE.Success] -> [ARC063.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #34
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC047.bam, ARC047.alignBowtiePE.Success] -> [ARC047.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #35
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC117.bam, ARC117.alignBowtiePE.Success] -> [ARC117.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #36
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC060.bam, ARC060.alignBowtiePE.Success] -> [ARC060.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #37
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC074.bam, ARC074.alignBowtiePE.Success] -> [ARC074.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #38
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC092.bam, ARC092.alignBowtiePE.Success] -> [ARC092.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #39
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC111.bam, ARC111.alignBowtiePE.Success] -> [ARC111.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #40
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC059.bam, ARC059.alignBowtiePE.Success] -> [ARC059.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #41
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC113.bam, ARC113.alignBowtiePE.Success] -> [ARC113.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #42
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC056.bam, ARC056.alignBowtiePE.Success] -> [ARC056.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #43
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC114.bam, ARC114.alignBowtiePE.Success] -> [ARC114.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #44
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC118.bam, ARC118.alignBowtiePE.Success] -> [ARC118.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #45
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC110.bam, ARC110.alignBowtiePE.Success] -> [ARC110.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #46
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC082.bam, ARC082.alignBowtiePE.Success] -> [ARC082.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #47
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC076.bam, ARC076.alignBowtiePE.Success] -> [ARC076.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #48
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC116.bam, ARC116.alignBowtiePE.Success] -> [ARC116.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #49
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC134.bam, ARC134.alignBowtiePE.Success] -> [ARC134.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found


Exception #50
exceptions.TypeError(coercing to Unicode: need string or buffer, list found):
for RedDog.checkBam.Job = [[ARC103.bam, ARC103.alignBowtiePE.Success] -> [ARC103.checkBam.Success]]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 1054, in checkBam
    runStageCheck('checkBam', flagFile, bamFile, readType, read_set)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/utils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
TypeError: coercing to Unicode: need string or buffer, list found

RMann24 avatar Dec 22 '15 04:12 RMann24

Sorry David, forgot to mention that .bam files have been produced in all of the /temp subfolders

Dr Rachel Mann | Research Scientist | Microbiology | Biosciences Research Division Department of Economic Development, Jobs, Transport and Resources AgriBio, Centre for AgriBioscience, 5 Ring Road, Bundoora VIC 3086
T: +61 3 9032 7280 | E: [email protected] W: Wednesday, Thursday and Friday

From: d-j-e [email protected] To: katholt/RedDog [email protected], Cc: RMann24 [email protected] Date: 22/12/2015 03:07 PM Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Hi Rachel, Can you check if the bams are actually being created? Check in the output/temp subfolders - unfortunately, the way the samtools part of the command works is to produce the bam file then populate it. So the pipeline is tricked into thinking the stage has successfully completed, even when it hasn't. Hence checkBam was added to the pipe - this simply checks that the bam file is at least a reasonable size. BTW the pipe doesn't actually produce a checkBam.pbs, instead it produces a series of tmp files to launch each job - are these being produced (in the reddog folder itself)? Can you also try run the pipe again so I can see the error? To capture the run just do: rubra RedDog --config your_config --style run > run.txt when you hit return, then hit 'y' and return and the pipe will run sending the output to the run.txt file. I'm working today and tomorrow before taking a break until early January, so hopefully we can fix this asap... cheers, David — Reply to this email directly or view it on GitHub.

Department of Economic Development, Jobs, Transport and Resources, Government of Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential information. If you are not the intended recipient, you may not distribute or reproduce this e-mail or the attachments. If you have received this message in error, please notify us by return email.

RMann24 avatar Dec 22 '15 04:12 RMann24

Oh dear, this appears to be a bug in the execution of a Rubra command - Bernie, one of the coauthors of RedDog, is also the main collaborator of Rubra so I'll have to check with him exactly what is going on... And I'm not sure if he's on holidays yet.

However, I'd also like for you to run the small test data described in the manual to see if the same thing happens with that data set if you have the time atm.

d-j-e avatar Dec 22 '15 05:12 d-j-e

Hi David, apologies in the delay in getting back to you. I have run the test set (see attached). It did not progress past making the output directory. The pipeline seems to intermittently progress to different points, even with the same data sets. We'd appreciate any help, there are a few groups here that would really like to use the pipeline,

kind regards,

Rachel

From: d-j-e [email protected] To: katholt/RedDog [email protected], Cc: RMann24 [email protected] Date: 22/12/2015 04:17 PM Subject: Re: [RedDog] Pipeline not progressing past checkBam (#45)

Oh dear, this appears to be a bug in the execution of a Rubra command - Bernie, one of the coauthors of RedDog, is also the main collaborator of Rubra so I'll have to check with him exactly what is going on... And I'm not sure if he's on holidays yet. However, I'd also like for you to run the small test data described in the manual to see if the same thing happens with that data set if you have the time atm. — Reply to this email directly or view it on GitHub.

Department of Economic Development, Jobs, Transport and Resources, Government of Victoria, Victoria, Australia.

This email, and any attachments, may contain privileged and confidential information. If you are not the intended recipient, you may not distribute or reproduce this e-mail or the attachments. If you have received this message in error, please notify us by return email.

RedDog V1beta.8 - phylogeny run

Copyright (c) 2016 David Edwards, Bernie Pope, Kat Holt All rights reserved. (see README.txt for more details)

Mapping: Bowtie2 V2.2.3 Preset Option: --sensitive-local 2 replicon(s) in GenBank reference reference 2 replicon(s) to be reported 3 sequence pair(s) to be mapped

Output folder: /group/pathogens/Rachel/reddog_test_output/

Start Pipeline? (y/n) Starting pipeline... 93 jobs to be executed in total

Traceback (most recent call last): File "/usr/local/bin/rubra", line 9, in load_entry_point('Rubra==0.1.5', 'console_scripts', 'rubra')() File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/rubra .py", line 66, in main gnu_make_maximal_rebuild_mode=rebuildMode) File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 2680, in pi peline_run raise errt ruffus.ruffus_exceptions.RethrownJobError:

Exceptions running jobs for

'def RedDog.makeDir(...):'

Original exception:

Exception #1
exceptions.IOError([Errno 2] No such file or directory: '/group/pathogens/Ra                                       chel/reddog_test_output/temp/success/dir.makeDir.Success'):
for RedDog.makeDir.Job = [False -> dir.makeDir.Success]

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 517, in                                        run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_clea                                       nup, touch_files_only)
  File "/usr/local/lib/python2.7/site-packages/ruffus/task.py", line 447, in                                        job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "RedDog.py", line 950, in makeDir
    runStageCheck('makeDir', flagFile, outPrefix, full_sequence_list_string)
  File "/usr/local/lib/python2.7/site-packages/Rubra-0.1.5-py2.7.egg/rubra/u                                       tils.py", line 130, in runStageCheck
    open(flag_file, 'w').close()
IOError: [Errno 2] No such file or directory: '/group/pathogens/Rachel/reddo                                       g_test_output/temp/success/dir.makeDir.Success'

RMann24 avatar Feb 08 '16 03:02 RMann24

How are you editing the config file? - can you send me a copy to davide at unimelb dot edu dot au ?

d-j-e avatar Feb 08 '16 07:02 d-j-e

Hi Rachel, I did notice a couple of things with the config file you are using - The config files are both for version 1b5, but you are now running version 1b8 - try changing to the new version of config file (you may have to get Jason's help to include his "local hacks"). I also noticed a typo that may not be helping... the following bit ends with a comma (this list will be slightly different to the one for v1b5, because we have turned back to Fasttree rather than raxml (you might have to install fasttree as well...):

}
stageDefaults = {
    "distributed": True,
    "walltime": "01:00:00",
    "memInGB": 4,
    "queue": 'batch',
    "modules": [
        "bwa",
        "samtools",
        "bcftools",
        "gcc",
        "bowtie2",
        "eautils",


    ]
}

The final entry in the modules list should not have a comma ie.

       "bowtie2",
        "eautils"
    ]
}

Have a go at those changes and let me know how it goes... If you are really keen to install this and still have problems, I may have to volunteer to visit you all (Did my undergrad at La Trobe many years ago...).

d-j-e avatar Feb 08 '16 23:02 d-j-e