High false positive number and x-replacement simulation failure

[pixuenan]

Hello,

I'm running ximmer on 15 exome samples from 1000G by docker.

Here is my config file

title="Test Simulation 15 samples"
bam_files=[
    "/data/wes/run/20181205/02.bam/HG01200/HG01200_recal.bam",
    ......
        ]
target_regions="/data/wes/Ref/bed/SeqCap_EZ_Exome_v3_hg19_capture_targets.bed"
simulation_type='downsample'

known_cnvs="/data/wes/cnv/00.simulation/15sample_cnv.tsv"

samples {
males = [
          "HG01200",
          .....,
          ]
  females = [
          "HG00743",
          ......,]
}
regions=8..20
deletionsPerSample=10
runs=5

callers {
        xhmm {}
        exomedepth {}
}

The used command line was

docker run -m 50G --rm -i --log-driver=none \
-e XIMMER_REF=/reference/Database/WES/GATK_bundle/hg19/fasta/ucsc.hg19.fasta \
-v /data/:/reference -v /data/:/data ximmer:latest ximmer \
-c /data/wes/cnv/00.simulation/config.groovy.1 -v \
-o /data/wes/cnv/00.simulation/output.docker \
> /data/wes/cnv/00.simulation/run.log 2>&1

However, while only 140 deletions were simulated, 9546 CNV were reported. Is there any explanation of the high false positive number?

[pixuenan]

And an error happened when I simulated the data with X-replacement method. The config file

title="Test Simulation 15 samples X-replacement"
bam_files=[
    "/data/wes/run/20181205/02.bam/HG01200/HG01200_recal.bam",
    ......
        ]
target_regions="/data/wes/Ref/bed/SeqCap_EZ_Exome_v3_hg19_capture_targets.bed"
simulation_type='replace'
concurrency=20

known_cnvs="/data/wes/cnv/00.simulation/15sample_cnv.tsv"

samples {
  males = [
          "HG01200",
          ......
          ]
  females = [
          "HG00743",
          ......
          ]

}

regions=2..10
deletionsPerSample=10
runs=3
callers {
        xhmm {}
        exomedepth {}
}

The error

CNVSimulator	[27]	INFO	|4:18:22 Creating /data/wes/cnv/00.simulation/output.docker.x_replace/run1/bams/HG01089_chrX_19934015-19948354.bam using replace mode 
CNVSimulator	[27]	INFO	|4:18:22 Querying source reads from Male alignment ... 
Processed 572 in 0.004 seconds  @ 95333.34/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=285)
Processed 1214 in 0.005 seconds  @ 173428.58/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=602)
Processed 1302 in 0.006 seconds  @ 162750.00/s   (HG00742 chrX, Spool Size=13, Buffer Size=13, ForcedQueries=0, Pairs=641)
Processed 834 in 0.004 seconds  @ 139000.00/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=415)
Processed 531 in 0.003 seconds  @ 106200.00/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=264)
Processed 1049 in 0.005 seconds  @ 149857.14/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=521)
Processed 1069 in 0.004 seconds  @ 178166.67/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=527)
Processed 134 in 0.001 seconds  @ 44666.67/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=67)
Processed 585 in 0.003 seconds  @ 117000.00/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=291)
Processed 443 in 0.003 seconds  @ 88600.00/s   (HG00742 chrX, Spool Size=0, Buffer Size=0, ForcedQueries=0, Pairs=221)
CNVSimulator	[27]	INFO	|4:18:22 Found 3834 read pairs in region over male alignment HG00742 
Ximmer	[27]	SEVERE	|4:18:22 Sample HG01089 failed in simulation 
groovy.lang.MissingMethodException: No signature of method: gngs.SAM.filterOrderedPairs() is applicable for argument types: (java.util.LinkedHashMap, java.util.LinkedHashMap, java.lang.Strin
g, CNVSimulator$_createBamByReplacement_closure9) values: [[end:CNVSimulator$_createBamByReplacement_closure8@1457a775, ...], ...]

[ssadedin]

Sorry about that! It looks like another fix pulled in an updated API for a dependency, so I needed to make a tweak for that code. I've pushed a change (9d165ea69e66aa91fcaedece2218b211449c1386) which should fix it. If you can do git pull and try again, that would be great.

Thanks!

[pixuenan]

Hi Simon,

I rebuild the docker images with the git pull, now the x replacement can start successfully, but I got another error

Sat Dec 29 00:54:14 UTC 2018 :   Processed 13927000 @ 38357.80/s Written 9466981, Buffered 101 RefIndex=7
Processed 33086500     @ 89296.42/s HG01089 null        chr7, Spooled=0, [:] Preproc=0 resolveQueue=0, Lookaheads=0,0 Unpaired=0, ChimericBuffer=0 Forced=0, Async=0/0 Pairs=16346852
Processed 16630000     14283288 reads written to /data/data_pixn/wes/cnv/00.simulation/output.docker.x_replace/run1/bams/HG01089_chrX_47098209-47101798.bam
Sat Dec 29 00:54:24 UTC 2018 :   Processed 14244500 @ 31724.62/s Written 9694377, Buffered 31 RefIndex=7
Processed 33838500     @ 75162.42/s HG01089 143558111   chr7, Spooled=484, [pending:9, resolved:91] Preproc=0 resolveQueue=0, Lookaheads=0,0 Unpaired=0, ChimericBuffer=0 Forced=0, Async=0/0 
Pairs=16718123
Sat Dec 29 00:54:33 UTC 2018 :   Processed 0unresolvable=0
Sat Dec 29 00:54:33 UTC 2018 :   Processed 0unresolvable=0
Sat Dec 29 00:54:33 UTC 2018 :   Processed 0unresolvable=0
gngs.RegionScan [35]    INFO    |12:54:33 3807/4692 reads were not paired after expected resolution 
Sat Dec 29 00:54:34 UTC 2018 :   Processed 0unresolvable=8
gngs.RegionScan [33]    INFO    |12:54:34 2598/4712 reads were not paired after expected resolution 
An exception occurred in the Actor thread Actor Thread 2
java.lang.IllegalArgumentException: Alignments added out of order in SAMFileWriterImpl.addAlignment for file:///data/data_pixn/wes/cnv/00.simulation/output.docker.x_replace/run1/bams/HG01089
_chrX_47098209-47101798.bam. Sort order is coordinate. Offending records are at [chr8:7116676] and [chr8:7116524]
        at htsjdk.samtools.SAMFileWriterImpl.assertPresorted(SAMFileWriterImpl.java:213)
        at htsjdk.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:200)
        at gngs.OrderedPairWriter.addAlignmentPair(OrderedPairWriter.groovy:83)
        at gngs.OrderedPairActor$_act_closure1$_closure2.doCall(OrderedPairActor.groovy:58)
        at gngs.OrderedPairActor$_act_closure1$_closure2.call(OrderedPairActor.groovy)
        at groovyx.gpars.actor.DefaultActor.onMessage(DefaultActor.java:94)

May I know is there any module or package missed in my docker that resulted in this error?

[pixuenan]

After I removed the sample HG01089, everything worked.