Error in merge.data.table() - AberrantExpression_pipeline_OUTRIDER_pvalsOutrider_R

[canankolakoglu]

Hi, I'm using drop 1.3.3 in order to do aberrant expression analysis. However during outrider execution with 2 bam data and 9 external counts, I'm getting the following error:

[Tue May 30 16:17:50 2023]
Finished job 15.
11 of 18 steps (61%) done
Select jobs to execute...

[Tue May 30 16:17:50 2023]
rule AberrantExpression_pipeline_OUTRIDER_pvalsOutrider_R:
    input: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods_fitted.Rds, /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_data/preprocess/v38/gene_name_mapping_v38.tsv, Scripts/AberrantExpression/pipeline/OUTRIDER/pvalsOutrider.R
    output: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods.Rds
    log: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/.drop/tmp/AE/v38/outrider/pvalsOUTRIDER.Rds
    jobid: 14
    reason: Missing output files: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods.Rds; Input files updated by another job: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods_fitted.Rds, /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_data/preprocess/v38/gene_name_mapping_v38.tsv
    wildcards: annotation=v38, dataset=outrider
    threads: 10
    resources: tmpdir=/tmp

Warning messages:
1: package ‘GenomeInfoDb’ was built under R version 4.2.3 
2: package ‘ggplot2’ was built under R version 4.2.3 
3: package ‘dplyr’ was built under R version 4.2.3 
Error in merge.data.table(data.table(gene_name = genes), gene_annot_dt[,  : 
  Elements listed in `by` must be valid column names in x and y
Calls: convert_to_geneIDs ... lapply -> FUN -> merge -> merge -> merge.data.table
In addition: Warning message:
In merge.data.table(data.table(gene_name = genes), gene_annot_dt[,  :
  You are trying to join data.tables where 'x' argument is 0 columns data.table.
Execution halted
[Tue May 30 16:18:04 2023]
Error in rule AberrantExpression_pipeline_OUTRIDER_pvalsOutrider_R:
    jobid: 14
    input: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods_fitted.Rds, /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_data/preprocess/v38/gene_name_mapping_v38.tsv, Scripts/AberrantExpression/pipeline/OUTRIDER/pvalsOutrider.R
    output: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/output/processed_results/aberrant_expression/v38/outrider/outrider/ods.Rds
    log: /mnt/NAS_DEPO_2/OUR_PROJECTS/1501_3210640_2020/OUTPUTS/RNA-Seq/270223/drop-outrider_1.3.3/.drop/tmp/AE/v38/outrider/pvalsOUTRIDER.Rds (check log file(s) for error details)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-05-30T144827.863094.snakemake.log 

Here my config.yaml (https://github.com/gagneurlab/drop/files/11622184/config.yaml.txt) file and sample_annotation (https://github.com/gagneurlab/drop/files/11622187/sample_annotation.tsv.txt) table Also you can find 2023-05-30T144827.863094.snakemake.log (https://github.com/gagneurlab/drop/files/11622192/2023-05-30T144827.863094.snakemake.log.txt) file

Thanks.

[vyepez88]

Hi, can you verify that the gtf file used for the external counts is the same as for the local samples?

[canankolakoglu]

Hi again, Sure, here the config.yaml (https://github.com/gagneurlab/drop/files/11622243/config.yaml.txt) and sample_annotation(https://github.com/gagneurlab/drop/files/11622249/sample_annotation.tsv.txt) files that are used to create external counts.

[ischeller]

Hi @canankolakoglu , sorry for the late follow up. From your error, this does not seem to be related to the external counts, but rather the use or not-use of subsets of genes during FDR correction. To figure out what is going on, can you run the AberrantExpression_pipeline_OUTRIDER_pvalsOutrider_R script interactively until this line and report what the value of subsets is at this point? Given your config.yaml file with genesToTest: null (i.e. not using such subsets), the value should be NULL