DROP version 1.2.1 Issues with running aberrant Expression and Splicing Modules on Own Data

[lemdcock]

Dear Gagneur Lab,

I recently tried to run the latest version of your DROP pipeline on our own data. I, unfortunately, ran into some issues. Our data is aligned to hg38 with Hisat2 using Ensembl version 105 as annotation during the generation of the Hisat2 indices. When we tried to run DROP version 1.2.1 on our data, we got the following issues with the aberrant Expression and Splicing modules:

aberrantExpression Module:

[Mon Jul 25 16:23:11 2022]
rule AberrantExpression_pipeline_Counting_mergeCounts_R:
    input: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/count_ranges.Rds, /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/params/merge/OutRider_mergeParams.csv, Scripts/AberrantExpression/pipeline/Counting/mergeCounts.R
    output: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/outrider/OutRider/total_counts.Rds
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.drop/tmp/AE/Ensembl_v105/OutRider/merge.Rds
    jobid: 5
    reason: Missing output files: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/outrider/OutRider/total_counts.Rds; Input files updated by another job: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/count_ranges.Rds
    wildcards: annotation=Ensembl_v105, dataset=OutRider
    resources: tmpdir=/tmp

read 0 files
Error in seq_len(ncol(a1)) :
  argument must be coercible to non-negative integer
Calls: SummarizedExperiment -> DataFrame
In addition: Warning message:
In seq_len(ncol(a1)) : first element used of 'length.out' argument
Execution halted
[Mon Jul 25 16:23:17 2022]
Error in rule AberrantExpression_pipeline_Counting_mergeCounts_R:
    jobid: 5
    output: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_expression/Ensembl_v105/outrider/OutRider/total_counts.Rds
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.drop/tmp/AE/Ensembl_v105/OutRider/merge.Rds (check log file(s) for error message)

RuleException:
CalledProcessError in line 57 of /tmp/tmpc5vp4g2j:
Command 'set -euo pipefail;  Rscript --vanilla /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.snakemake/scripts/tmpi663gd9x.mergeCounts.R' returned non-zero exit status 1.
  File "/tmp/tmpc5vp4g2j", line 57, in __rule_AberrantExpression_pipeline_Counting_mergeCounts_R
  File "/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2022-07-25T162113.227654.snakemake.log

aberrantSplicing Module:

[Mon Jul 25 17:06:03 2022]
rule AberrantSplicing_pipeline_Counting_01_1_countRNA_splitReads_samplewise_R:
    input: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_splicing/datasets/cache/raw-local-Fraser/fds.done, Scripts/AberrantSplicing/pipeline/Counting/01_1_countRNA_splitReads_samplewise.R
    output: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_splicing/datasets/cache/raw-local-Fraser/sample_tmp/splitCounts/sample_RNA013918.done
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.drop/tmp/AS/Fraser/splitReads/RNA013918.Rds
    jobid: 28
    reason: Missing output files: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_splicing/datasets/cache/raw-local-Fraser/sample_tmp/splitCounts/sample_RNA013918.done; Input files updated by another job: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_splicing/datasets/cache/raw-local-Fraser/fds.done
    wildcards: dataset=Fraser, sample_id=RNA013918
    resources: tmpdir=/tmp

Load packages
Loading required package: rtracklayer
Mon Jul 25 17:06:14 2022: Count split reads for sample: RNA013918
Error in asMethod(object) :
  The character vector to convert to a GRanges object must contain
  strings of the form "chr:start-end" or "chr:start-end:strand", with end
  >= start - 1, or "chr:pos" or "chr:pos:strand". For example:
  "chr1:2501-2900", "chr1:2501-2900:+", or "chr1:740". Note that ".." is
  a valid alternate start/end separator. Strand can be "+", "-", "*", or
  missing.
Calls: countSplitReads -> GRanges -> as -> asMethod
Execution halted
[Mon Jul 25 17:06:14 2022]
Error in rule AberrantSplicing_pipeline_Counting_01_1_countRNA_splitReads_samplewise_R:
    jobid: 28
    output: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/work/processed_data/aberrant_splicing/datasets/cache/raw-local-Fraser/sample_tmp/splitCounts/sample_RNA013918.done
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.drop/tmp/AS/Fraser/splitReads/RNA013918.Rds (check log file(s) for error message)

RuleException:
CalledProcessError in line 36 of /tmp/tmpyf2f5ef3:
Command 'set -euo pipefail;  Rscript --vanilla /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_1/.snakemake/scripts/tmp7h_lw1x0.01_1_countRNA_splitReads_samplewise.R' returned non-zero exit status 1.
  File "/tmp/tmpyf2f5ef3", line 36, in __rule_AberrantSplicing_pipeline_Counting_01_1_countRNA_splitReads_samplewise_R
  File "/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2022-07-25T170514.649929.snakemake.log

Based on your recommendations in your Nature Methods paper, we hypothesise that these problems could arise due to our use of Ensembl and Hisat2 instead of Gencode and STAR. Could you confirm that this is the most likely cause of our issues?

Kind regards, Laurenz De Cock

[schance995]

I am running DROP on Gencode and STAR data. At least for aberrantExpression, I get the read 0 files error when I don't include external counts. I think it is because of drop/config/submodules/AberrantExpression.py:

def getCountFiles(self, annotation, group):
    """                                                                                                       
    Get all count files from DROP (counted from BAM file) and external count matrices                         
    :param annotation: annotation name from wildcard                                                          
    :param group: DROP group name from wildcard                                                               
    :return: list of files                                                                                    
    """

    # if sample annotation table does not contain GENE_COUNTS_FILE column. return no external counts          
    if("GENE_COUNTS_FILE" not in self.sampleAnnotation.SAMPLE_ANNOTATION_COLUMNS):
        return []

When GENE_COUNTS_FILE is not included because the analysis does not use external counts, then this function does not return any files.

    bam_IDs = self.sampleAnnotation.getIDsByGroup(group, assay="RNA")
    file_stump = self.processedDataDir / "aberrant_expression" / annotation / "counts" / "{sampleID}.Rds"
    count_files = expand(str(file_stump), sampleID=bam_IDs)
    extCountFiles = self.sampleAnnotation.getImportCountFiles(annotation, group, file_type="GENE_COUNTS_FILE")
    count_files.extend(extCountFiles)
    return count_files

If GENE_COUNTS_FILE is present but all columns are empty then the function crashes at the assignment of extCountFiles.

Either way, I don't think DROP can currently run aberrant expression without external counts. The manuscript states that DROP should support running without external counts, so hopefully this gets fixed.

[nickhsmith]

Sorry you are having issues. It seems there are 2 separate issues here.

1. @lemdcock there seems to be a difference in formatting within the STAR aligned BAM files and the HISAT2 alignment, I know we haven't done an extensive test on bam files that are generated using other alignment strategies. But this could very well be the case. @vyepez88 have you had any experience with other alignment tools besides STAR?

2. @schance995 The solution for experiments without external counts is to delete the column GENE_COUNTS_FILE then it will run.

We will handle the handling of an empty column to correctly ignore the optional external count files and will push this fix ASAP to handle an empty GENE_COUNTS_FILE or another optional column.

[nickhsmith]

UPDATE: Actually deleting this column is not sufficient. You must build a dummy external file. This is our error, we will push a bug fix ASAP.

[nickhsmith]

This should be fixed in the newest update. You can either install directly using git or update a conda environment

conda update drop
cd path/to/working/dir
drop update

Thank you for your help finding this bug and patience while we hopefully resolved it!

[schance995]

1.2.2 works for me

[nickhsmith]

@lemdcock any luck on your side?

[lemdcock]

Dear @nickhsmith,

I was on holiday the past week, but I will test the new version as soon as possible.

In the meantime my colleague, Sarah Vergult, ran the aberrantSplicing module on our data after realignment with STAR and using GENCODE v41 as annotation. Unfortunately, we ran into an error again.

In the snake log file, we get the following information:

[Mon Aug  8 08:49:38 2022]
rule AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R:
    input: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/predictedMeans_theta.h5, Scripts/AberrantSplicing/pipeline/FRASER/06_calculation_stats_AE_FraseR.R
    output: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.drop/tmp/AS/fraser/06_stats.Rds
    jobid: 88
    reason: Missing output files: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5; Input files updated by another job: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/predictedMeans_theta.h5
    wildcards: dataset=fraser
    threads: 12
    resources: tmpdir=/tmp

Load packages
Loading assay: rawCountsJ
Loading assay: psi5
Loading assay: psi3
Loading assay: rawOtherCounts_psi5
Loading assay: rawOtherCounts_psi3
Loading assay: delta_psi5
Loading assay: delta_psi3
Loading assay: predictedMeans_psi5
Loading assay: predictedMeans_psi3
Loading assay: rawCountsSS
Loading assay: theta
Loading assay: rawOtherCounts_theta
Loading assay: delta_theta
Loading assay: predictedMeans_theta
Warning messages:
1: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
2: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
3: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
4: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
5: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
6: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
Mon Aug  8 08:51:47 2022: Preparing data for HDF5 conversion: zScores_psi5
Mon Aug  8 08:51:47 2022: Writing data: zScores_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/zScores_psi5.h5
Error in calculatePvalues(fds, type = type) :
  lazy-load database '/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/R/library/FRASER/R/FRASER.rdb' is corrupt
In addition: There were 22 warnings (use warnings() to see them)
Execution halted
[Mon Aug  8 16:42:48 2022]
Error in rule AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R:
    jobid: 88
    output: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.drop/tmp/AS/fraser/06_stats.Rds (check log file(s) for error message)

RuleException:
CalledProcessError in line 224 of /tmp/tmp8srwew6r:
Command 'set -euo pipefail;  Rscript --vanilla /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.snakemake/scripts/tmpogjvges5.06_calculation_stats_AE_FraseR.R' returned non-zero exit status 1.
  File "/tmp/tmp8srwew6r", line 224, in __rule_AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R
  File "/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2022-08-05T225001.868509.snakemake.log

I also checked the 06_stats.Rds log-file. Unfortunately I did not find any additional information in this file.

Do you potentially have an idea what is going wrong during this run? Why did we get a corrupt database?

Kind regards, Laurenz De Cock

[nickhsmith]

Seems like the FRASER install didn't work correctly:

lazy-load database '/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/R/library/FRASER/R/FRASER.rdb' is corrupt

could you please try to uninstall and reinstall that?

On Mon, Aug 8, 2022 at 5:32 PM lemdcock @.***> wrote:

Dear @nickhsmith https://github.com/nickhsmith,

I was on holiday the past week, but I will test the new version as soon as possible.

In the meantime my colleague, Sarah Vergult, ran the aberrantSplicing module on our data after realignment with STAR and using GENCODE v41 as annotation. Unfortunately, we ran into an error again.

In the snake log file, we get the following information:

[Mon Aug 8 08:49:38 2022] rule AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R: input: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/predictedMeans_theta.h5, Scripts/AberrantSplicing/pipeline/FRASER/06_calculation_stats_AE_FraseR.R output: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5 log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.drop/tmp/AS/fraser/06_stats.Rds jobid: 88 reason: Missing output files: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5; Input files updated by another job: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/predictedMeans_theta.h5 wildcards: dataset=fraser threads: 12 resources: tmpdir=/tmp

Load packages Loading assay: rawCountsJ Loading assay: psi5 Loading assay: psi3 Loading assay: rawOtherCounts_psi5 Loading assay: rawOtherCounts_psi3 Loading assay: delta_psi5 Loading assay: delta_psi3 Loading assay: predictedMeans_psi5 Loading assay: predictedMeans_psi3 Loading assay: rawCountsSS Loading assay: theta Loading assay: rawOtherCounts_theta Loading assay: delta_theta Loading assay: predictedMeans_theta Warning messages: 1: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' 2: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' 3: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' 4: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' 5: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' 6: In checkReadType(x, type) : Read type was not specified!We will assume the default: 'j' Mon Aug 8 08:51:47 2022: Preparing data for HDF5 conversion: zScores_psi5 Mon Aug 8 08:51:47 2022: Writing data: zScores_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/zScores_psi5.h5 Error in calculatePvalues(fds, type = type) : lazy-load database '/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/R/library/FRASER/R/FRASER.rdb' is corrupt In addition: There were 22 warnings (use warnings() to see them) Execution halted [Mon Aug 8 16:42:48 2022] Error in rule AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R: jobid: 88 output: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/Output/processed_data/aberrant_splicing/datasets/savedObjects/fraser/padjBetaBinomial_theta.h5 log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.drop/tmp/AS/fraser/06_stats.Rds (check log file(s) for error message)

RuleException: CalledProcessError in line 224 of /tmp/tmp8srwew6r: Command 'set -euo pipefail; Rscript --vanilla /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_STAR_GENCODE/.snakemake/scripts/tmpogjvges5.06_calculation_stats_AE_FraseR.R' returned non-zero exit status 1. File "/tmp/tmp8srwew6r", line 224, in __rule_AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R File "/kyukon/scratch/gent/vo/001/gvo00101/vsc42698/miniconda2/envs/drop_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: .snakemake/log/2022-08-05T225001.868509.snakemake.log

I also checked the 06_stats.Rds log-file. Unfortunately I did not find any additional information in this file.

Do you potentially have an idea what is going wrong during this run? Why did we get a corrupt database?

Kind regards, Laurenz De Cock

— Reply to this email directly, view it on GitHub https://github.com/gagneurlab/drop/issues/347#issuecomment-1208281575, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABWP745XRVEHLCX6S4CWWT3VYER7FANCNFSM54VKXXQA . You are receiving this because you were mentioned.Message ID: @.***>

[lemdcock]

Dear @nickhsmith,

Drop version 1.2.2 worked for me as well.

I've reinstalled FRASER and am now running the aberrantSplicing module again. I'll keep you posted on the results.

Thank you for all your help the past couple of days.

Kind regards, Laurenz De Cock

[vyepez88]

great to hear that Laurenz! I'll close the issue once you confirm us that the module ran through.

[lemdcock]

Dear @vyepez88,

The current run of the aberrantSplicing module is finished. We did not run into any errors anymore, however, unfortunately the module did not fully run through either. There is a time limit of 72h on our HPC system and the module was not able to run through during this time period. Is this normal? We currently have 38 samples, that were aligned with STAR using GENCODE v41 as annotation.

Log-file output:

[Fri Aug 12 18:51:08 2022]
rule AberrantSplicing_pipeline_FRASER_06_calculation_stats_AE_FraseR_R:
    input: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/predictedMeans_theta.h5, Scripts/AberrantSplicing/pipeline/FRASER/06_calculation_stats_AE_FraseR.R
    output: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/padjBetaBinomial_theta.h5
    log: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/.drop/tmp/AS/Fraser/06_stats.Rds
    jobid: 90
    reason: Missing output files: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/padjBetaBinomial_theta.h5; Input files updated by another job: /data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/predictedMeans_theta.h5
    wildcards: dataset=Fraser
    threads: 20
    resources: tmpdir=/tmp

Load packages
Loading assay: rawCountsJ
Loading assay: psi5
Loading assay: psi3
Loading assay: rawOtherCounts_psi5
Loading assay: rawOtherCounts_psi3
Loading assay: delta_psi5
Loading assay: delta_psi3
Loading assay: predictedMeans_psi5
Loading assay: predictedMeans_psi3
Loading assay: rawCountsSS
Loading assay: theta
Loading assay: rawOtherCounts_theta
Loading assay: delta_theta
Loading assay: predictedMeans_theta
Warning messages:
1: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
2: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
3: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
4: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
5: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
6: In checkReadType(x, type) :
  Read type was not specified!We will assume the default: 'j'
Fri Aug 12 18:53:07 2022: Preparing data for HDF5 conversion: zScores_psi5
Fri Aug 12 18:53:07 2022: Writing data: zScores_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/zScores_psi5.h5
Sat Aug 13 02:52:33 2022: Preparing data for HDF5 conversion: rawCountsJ
Sat Aug 13 02:52:33 2022: Writing data: rawCountsJ to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/rawCountsJ.h5
Sat Aug 13 02:52:42 2022: Preparing data for HDF5 conversion: rawOtherCounts_psi5
Sat Aug 13 02:52:42 2022: Writing data: rawOtherCounts_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/rawOtherCounts_psi5.h5
Sat Aug 13 02:53:04 2022: Preparing data for HDF5 conversion: pvaluesBetaBinomial_junction_psi5
Sat Aug 13 02:53:04 2022: Writing data: pvaluesBetaBinomial_junction_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/pvaluesBetaBinomial_junction_psi5.h5
Sat Aug 13 03:17:01 2022: Preparing data for HDF5 conversion: pvaluesBetaBinomial_psi5
Sat Aug 13 03:17:01 2022: Writing data: pvaluesBetaBinomial_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/pvaluesBetaBinomial_psi5.h5
Sat Aug 13 03:17:32 2022: Preparing data for HDF5 conversion: padjBetaBinomial_psi5
Sat Aug 13 03:17:32 2022: Writing data: padjBetaBinomial_psi5 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/padjBetaBinomial_psi5.h5
Sat Aug 13 03:19:22 2022: Preparing data for HDF5 conversion: zScores_psi3
Sat Aug 13 03:19:22 2022: Writing data: zScores_psi3 to file: /kyukon/data/gent/shared/001/gvo00101/DROP/NVQ_Run093-NVQ_Run406_DROP_v1_2_2/work/processed_data/aberrant_splicing/datasets/savedObjects/Fraser/zScores_psi3.h5
slurmstepd: error: *** JOB 45303494 ON node3403.kirlia.os CANCELLED AT 2022-08-13T07:35:53 DUE TO TIME LIMIT ***

Kind regards, Laurenz De Cock

[vyepez88]

Dear Laurenz,

I don't understand :) At first, you mention that the current run is finished, but then that it didn't run through. 72h should be enough to run the full module though. Do you have the option filter=TRUE? Can you check in the Summary html how many junctions do you have? 8h for the calculation_stats_AE_FraseR step sounds like a lot, but not necessarily unreal. I guess simply restarting the module should work, as snakemake should realize the pending steps and proceed from there.

Best, Vicente

[nickhsmith]

@lemdcock Have you had any luck with the calculations? Thanks for your patience.

[nickhsmith]

@lemdcock Unless you have more problems I will close this issue for now.