fritzsedlazeck
Usage recommendations, confusion about mosdepth example command
Hi,
I am currently evaluating Spectre for CNV calling on some GIAB HG002 ONT data and I encountered a few issues which I wasn't able to resolve myself.
Using mosdepth I tried to create required regions.bed.gz file for use with Spectre, which worked fine with the following: mosdepth -t {threads} -x -b 1000 -Q 20 "variants/{wildcards.sample}/spectre/mosdepth/{wildcards.sample}"' "{input.bam}" 2>{log}' But, I don't understand why the "-c X" parameter is included in your example command. This would lead to only Chromosome X being evaluated on read depth right? Could you explain why this parameter is there?
When I tried to run Spectre itself I only got some CNVs on chrX and not on any autosomes or other contigs (I don't know the known CNVs for HG002 atm, but was surprised that I only got this result). Any idea what's causing this? The logs mention something about expecting trouble as the Minimum CNV length is smaller than 1000000 base pairs.
spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Spectre input: CNVCaller --coverage variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz --sample-id PAO89685_90Gbp --output-dir variants/PAO89685_90Gbp/spectre/ --reference /mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --blacklist /mnt/shared/tools/spectre-0.2.1/data/grch38_blacklist_spectre.bed --metadata /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Spectre version: 0.2.1 spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Starting spectre spectre::2024-08-16 08:56:06,869::INFO::spectre.spectre> Evaluate if a new .mdr file needs to be created spectre::2024-08-16 08:56:06,992::INFO::spectre.spectre> Using existing metadata file /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr spectre::2024-08-16 08:56:07,053::INFO::spectre.spectreCNV> Spectre calculating for: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz spectre::2024-08-16 08:56:07,054::INFO::spectre.spectreCNV> Data normalization and outlier removal (right tail) [W::hts_idx_load3] The index file is older than the data file: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz.csi [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? spectre::2024-08-16 08:56:07,630::INFO::spectre.analysis.analysis> Determined bin size from Mosdepth coverage: 1000 spectre::2024-08-16 08:56:07,692::WARNING::spectre.analysis.analysis> Minimum CNV length is smaller then 1000000 base pairs spectre::2024-08-16 08:56:07,692::WARNING::spectre.analysis.analysis> Expect trouble! [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? [W::tbx_parse1] Coordinate <= 0 detected. Did you forget to use the -0 option? spectre::2024-08-16 08:56:12,241::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr1: 248957 spectre::2024-08-16 08:56:12,342::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr2: 242194 spectre::2024-08-16 08:56:12,437::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr3: 198296 spectre::2024-08-16 08:56:12,515::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr4: 190215 spectre::2024-08-16 08:56:12,592::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr5: 181539 spectre::2024-08-16 08:56:12,664::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr6: 170806 spectre::2024-08-16 08:56:12,733::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr7: 159346 spectre::2024-08-16 08:56:12,798::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr8: 145139 spectre::2024-08-16 08:56:12,858::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr9: 138395 spectre::2024-08-16 08:56:12,916::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr10: 133798 spectre::2024-08-16 08:56:12,969::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr11: 135087 spectre::2024-08-16 08:56:13,024::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr12: 133276 spectre::2024-08-16 08:56:13,077::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr13: 114365 spectre::2024-08-16 08:56:13,125::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr14: 107044 spectre::2024-08-16 08:56:13,169::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr15: 101992 spectre::2024-08-16 08:56:13,212::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr16: 90339 spectre::2024-08-16 08:56:13,251::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr17: 83258 spectre::2024-08-16 08:56:13,286::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr18: 80374 spectre::2024-08-16 08:56:13,318::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr19: 58618 spectre::2024-08-16 08:56:13,343::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr20: 64445 spectre::2024-08-16 08:56:13,369::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr21: 46710 spectre::2024-08-16 08:56:13,390::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr22: 50819 spectre::2024-08-16 08:56:13,412::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chrX: 156041 spectre::2024-08-16 08:56:13,479::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chrY: 57228 spectre::2024-08-16 08:56:13,505::WARNING::spectre.analysis.analysis> Chromosome chrY: no data available! spectre::2024-08-16 08:56:13,505::INFO::spectre.analysis.analysis> Number positions to be tested on chromosome chr16_KI270728v1_random: 1873 spectre::2024-08-16 08:56:38,297::INFO::spectre.analysis.analysis> Selected threshold for DEL=1.3083 and DUP=2.626 for a ploidy of 2 spectre::2024-08-16 08:56:38,326::INFO::spectre.spectreCNV> CNV calling - Coverage for sample: PAO89685_90Gbp spectre::2024-08-16 08:56:38,326::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr1 spectre::2024-08-16 08:56:38,844::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr2 spectre::2024-08-16 08:56:39,419::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr3 spectre::2024-08-16 08:56:39,919::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr4 spectre::2024-08-16 08:56:40,427::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr5 spectre::2024-08-16 08:56:40,937::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr6 spectre::2024-08-16 08:56:41,499::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr7 spectre::2024-08-16 08:56:41,979::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr8 spectre::2024-08-16 08:56:42,428::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr9 spectre::2024-08-16 08:56:42,817::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr10 spectre::2024-08-16 08:56:43,260::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr11 spectre::2024-08-16 08:56:43,739::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr12 spectre::2024-08-16 08:56:44,231::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr13 spectre::2024-08-16 08:56:44,613::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr14 spectre::2024-08-16 08:56:44,974::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr15 spectre::2024-08-16 08:56:45,298::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr16 spectre::2024-08-16 08:56:45,573::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr17 spectre::2024-08-16 08:56:45,867::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr18 spectre::2024-08-16 08:56:46,173::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr19 spectre::2024-08-16 08:56:46,413::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr20 spectre::2024-08-16 08:56:46,659::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr21 spectre::2024-08-16 08:56:46,806::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr22 spectre::2024-08-16 08:56:46,969::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chrX spectre::2024-08-16 08:56:47,905::INFO::spectre.analysis.analysis> Calculating CNVs for PAO89685_90Gbp @ chromosome chr16_KI270728v1_random spectre::2024-08-16 08:57:31,868::INFO::spectre.analysis.cnv_metrics> DEL border:1.3082655826558265 DUP border:2.6260162601626016 spectre::2024-08-16 08:57:32,358::INFO::spectre.analysis.analysis> refining cnv calls spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:57:44,589::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:57:57,730::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:07,912::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:17,715::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:27,048::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:27,049::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:35,837::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:43,679::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:50,723::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:58:56,104::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:02,721::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:09,832::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:16,911::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:22,050::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:22,051::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:26,546::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:30,397::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:33,459::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:36,942::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:40,770::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:43,788::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:46,763::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:48,312::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:49,993::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:52,800::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Estimated DEL threshold 0.9967970909191939 is under the allowed minimum threshold of 1. spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 1 spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Estimated DUP threshold 3.007193760298306 is over the allowed maximum threshold of 3. spectre::2024-08-16 08:59:52,845::WARNING::spectre.analysis.cnv_metrics> Using fallback to threshold of: 3 spectre::2024-08-16 08:59:52,845::INFO::spectre.spectreCNV> Calculate CNV metrics spectre::2024-08-16 09:00:20,053::INFO::spectre.analysis.cnv_metrics> DEL border:1.3082655826558265 DUP border:2.6260162601626016 spectre::2024-08-16 09:00:20,076::INFO::spectre.spectreCNV> Merge CNV and LoH candidates spectre::2024-08-16 09:00:20,076::DEBUG::spectre.classes.loh_candidate> CNV cand: 9 spectre::2024-08-16 09:00:20,076::INFO::spectre.spectreCNV> Final candidates are written to .spc file spectre::2024-08-16 09:00:26,716::INFO::spectre.spectreCNV> Results are writen to bed file spectre::2024-08-16 09:00:26,813::INFO::spectre.spectreCNV> Results are writen to VCF file spectre::2024-08-16 09:00:26,881::INFO::spectre.spectreCNV> Result plot in progress spectre::2024-08-16 09:00:27,034::INFO::spectre.plots.plot> CNVs in chromosome: chr1 spectre::2024-08-16 09:00:27,448::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr1.png spectre::2024-08-16 09:00:27,618::INFO::spectre.plots.plot> CNVs in chromosome: chr2 spectre::2024-08-16 09:00:28,002::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr2.png spectre::2024-08-16 09:00:28,064::INFO::spectre.plots.plot> CNVs in chromosome: chr3 spectre::2024-08-16 09:00:28,464::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr3.png spectre::2024-08-16 09:00:28,526::INFO::spectre.plots.plot> CNVs in chromosome: chr4 spectre::2024-08-16 09:00:28,893::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr4.png spectre::2024-08-16 09:00:28,954::INFO::spectre.plots.plot> CNVs in chromosome: chr5 spectre::2024-08-16 09:00:29,326::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr5.png spectre::2024-08-16 09:00:29,388::INFO::spectre.plots.plot> CNVs in chromosome: chr6 spectre::2024-08-16 09:00:29,759::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr6.png spectre::2024-08-16 09:00:29,819::INFO::spectre.plots.plot> CNVs in chromosome: chr7 spectre::2024-08-16 09:00:30,175::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr7.png spectre::2024-08-16 09:00:30,233::INFO::spectre.plots.plot> CNVs in chromosome: chr8 spectre::2024-08-16 09:00:30,563::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr8.png spectre::2024-08-16 09:00:30,621::INFO::spectre.plots.plot> CNVs in chromosome: chr9 spectre::2024-08-16 09:00:30,903::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr9.png spectre::2024-08-16 09:00:31,075::INFO::spectre.plots.plot> CNVs in chromosome: chr10 spectre::2024-08-16 09:00:31,382::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr10.png spectre::2024-08-16 09:00:31,441::INFO::spectre.plots.plot> CNVs in chromosome: chr11 spectre::2024-08-16 09:00:31,759::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr11.png spectre::2024-08-16 09:00:31,818::INFO::spectre.plots.plot> CNVs in chromosome: chr12 spectre::2024-08-16 09:00:32,159::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr12.png spectre::2024-08-16 09:00:32,241::INFO::spectre.plots.plot> CNVs in chromosome: chr13 spectre::2024-08-16 09:00:32,533::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr13.png spectre::2024-08-16 09:00:32,617::INFO::spectre.plots.plot> CNVs in chromosome: chr14 spectre::2024-08-16 09:00:32,877::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr14.png spectre::2024-08-16 09:00:32,966::INFO::spectre.plots.plot> CNVs in chromosome: chr15 spectre::2024-08-16 09:00:33,224::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr15.png spectre::2024-08-16 09:00:33,282::INFO::spectre.plots.plot> CNVs in chromosome: chr16 spectre::2024-08-16 09:00:33,531::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr16.png spectre::2024-08-16 09:00:33,589::INFO::spectre.plots.plot> CNVs in chromosome: chr17 spectre::2024-08-16 09:00:33,838::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr17.png spectre::2024-08-16 09:00:33,898::INFO::spectre.plots.plot> CNVs in chromosome: chr18 spectre::2024-08-16 09:00:34,286::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr18.png spectre::2024-08-16 09:00:34,344::INFO::spectre.plots.plot> CNVs in chromosome: chr19 spectre::2024-08-16 09:00:34,570::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr19.png spectre::2024-08-16 09:00:34,627::INFO::spectre.plots.plot> CNVs in chromosome: chr20 spectre::2024-08-16 09:00:34,875::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr20.png /usr/local/lib/python3.10/site-packages/spectre/plots/plot.py:39: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (
matplotlib.pyplot.figure) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParamfigure.max_open_warning). Consider usingmatplotlib.pyplot.close(). self.figure = plot_engine.figure(figsize=(8, 4)) spectre::2024-08-16 09:00:34,953::INFO::spectre.plots.plot> CNVs in chromosome: chr21 spectre::2024-08-16 09:00:35,140::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr21.png spectre::2024-08-16 09:00:35,218::INFO::spectre.plots.plot> CNVs in chromosome: chr22 spectre::2024-08-16 09:00:35,426::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr22.png spectre::2024-08-16 09:00:35,492::INFO::spectre.plots.plot> CNVs in chromosome: chrX spectre::2024-08-16 09:00:35,767::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chrX.png spectre::2024-08-16 09:00:35,823::INFO::spectre.plots.plot> CNVs in chromosome: chr16_KI270728v1_random spectre::2024-08-16 09:00:36,054::INFO::spectre.plots.plot> Plot saved: img/PAO89685_90Gbp_plot_cnv_chr16_KI270728v1_random.png spectre::2024-08-16 09:00:36,088::INFO::spectre.spectreCNV> Output dir: /mnt/scratch2/s.wielhouwer/ONT_variant_calling/pipeline/test_run/variants/PAO89685_90Gbp/spectre spectre::2024-08-16 09:00:36,088::INFO::spectre.spectreCNV> Done with sample PAO89685_90Gbp spectre::2024-08-16 09:00:36,088::INFO::spectre.spectre> Spectre finished
I am using GCA_000001405.15_GRCh38_no_alt_analysis_set.fna so I assume the metadata and blacklist are still compatible right? The contig names are identical (chr prefix) and it concerns hg38.
When including a snv file for marking LoH regions I get the following error.
spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Spectre input: CNVCaller --coverage variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz --sample-id PAO89685_90Gbp --output-dir variants/PAO89685_90Gbp/spectre/ --reference /mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --blacklist /mnt/shared/tools/spectre-0.2.1/data/grch38_blacklist_spectre.bed --metadata /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr --snv variants/PAO89685_90Gbp/PAO89685_90Gbp.snp.vcf.gz spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Spectre version: 0.2.1 spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Starting spectre spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Evaluate if a new .mdr file needs to be created spectre::2024-08-16 09:16:58,350::INFO::spectre.spectre> Using existing metadata file /mnt/shared/tools/spectre-0.2.1/data/grch38.mdr spectre::2024-08-16 09:16:58,362::INFO::spectre.spectreCNV> Spectre calculating for: variants/PAO89685_90Gbp/spectre/mosdepth/PAO89685_90Gbp.regions.bed.gz spectre::2024-08-16 09:16:58,363::INFO::spectre.spectreCNV> Analysing CN neutral state from SNV data spectre::2024-08-16 09:16:58,460::DEBUG::spectre.analysis.snv_analysis> Starting SNV Copy number state compute ... spectre::2024-08-16 09:16:58,460::DEBUG::spectre.analysis.snv_analysis> Het sites count per chromosome [W::vcf_parse] Contig '##cmdline=/opt/bin/run_clair3.sh --bam_fn=alignments/PAO89685_90Gbp.bam --ref_fn=/mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --platform=ont --threads=64 --model_path=/mnt/shared/tools/Clair3-1.0.10/container/models/r1041_e82_400bps_sup_v420 --output=variants/PAO89685_90Gbp/clair3' is not defined in the header. (Quick workaround: index the file with tabix.) [W::bcf_hrec_check] Invalid contig name: "##cmdline=/opt/bin/run_clair3.sh --bam_fn=alignments/PAO89685_90Gbp.bam --ref_fn=/mnt/shared/genomes/GRCh38/ncbi/GCA_000001405.15_GRCh38_no_alt_analysis_set.fna --platform=ont --threads=64 --model_path=/mnt/shared/tools/Clair3-1.0.10/container/models/r1041_e82_400bps_sup_v420 --output=variants/PAO89685_90Gbp/clair3" Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 161, in snv_nhet_chromosome [snv] = snv_record.samples.values() ValueError: not enough values to unpack (expected 1, got 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/bin/spectre", line 8, in
sys.exit(main()) File "/usr/local/lib/python3.10/site-packages/spectre/spectre.py", line 642, in main spectre_run.spectre_exe() File "/usr/local/lib/python3.10/site-packages/spectre/spectre.py", line 356, in spectre_exe spectre_main.cnv_call() File "/usr/local/lib/python3.10/site-packages/spectre/spectreCNV.py", line 71, in cnv_call self.snv_analysis.snv_copy_number_state() File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 202, in snv_copy_number_state use_chr_het_sites = self.snv_nhet_chromosome() # self.genome_info["chromosomes"] File "/usr/local/lib/python3.10/site-packages/spectre/analysis/snv_analysis.py", line 163, in snv_nhet_chromosome [snv] = snv_record.samples.values()[0] IndexError: list index out of range
Was this tool tested on Clair3 SNV data? Or should the falsely seen "##cmdline" contig be excluded using zgrep -v or similar to be able to use these vcf files?
I am using:
- mosdepth_0.3.8_build1
- spectre-0.2.1
- snf2json-0.1.0
- sniffles-2.4
- clair3-1.0.10
Btw, spectre should also work on other mammalian species right? Are there any ploidy limitations or such?
Best regards,
Serge
Hi @SergeWielhouwer, could you please share with us the header of the SNV-VCF file please. You can use the following command:
bcftools view --header-only file.vcf.gz > file_vcf_header.txt
We did test it with Clair3 for the most part (we did one test with Deep Variant) it seems that the header is not correctly formatted, but could be something else, lets check that first Spectre is mostly designed for humans, but, there is a ploidy parameter that can be used to change the default ploidy (2) for a given chromosome
Hi @SergeWielhouwer
Using mosdepth I tried to create required regions.bed.gz file for use with Spectre, which worked fine with the following: mosdepth -t {threads} -x -b 1000 -Q 20 "variants/{wildcards.sample}/spectre/mosdepth/{wildcards.sample}"' "{input.bam}" 2>{log}' But, I don't understand why the "-c X" parameter is included in your example command. This would lead to only Chromosome X being evaluated on read depth right? Could you explain why this parameter is there?
As you already correctly identified, the "-c X" does only get chromsome X, this should not be there. I will fix that ASAP in the README.
When I tried to run Spectre itself I only got some CNVs on chrX and not on any autosomes or other contigs (I don't know the known CNVs for HG002 atm, but was surprised that I only got this result). Any idea what's causing this? The logs mention something about expecting trouble as the Minimum CNV length is smaller than 1000000 base pairs.
There was a typo in the previous version, which was updated in the latest version on GitHub to 100kb. However, we did not push those changes to pip yet, since we considered them to be minor issues. We will push those changes with the next version of Spectre, which is currently under development. If you want the latest version or bug fixes, please clone and download the latest GitHub version. You can install Spectre locally via pip.
Regarding the minimum CNV length itself, we do not recommend going lower than 100kb. Reason being, Spectre was designed to detect large CNVs. Additionally, it depends on how high your signal-to-noise ratio is. Either mapping artifacts or just bad samples could introduce FPs.
Best, Philippe
Hi,
Thanks for letting me know and the quick fix! Please find the vcf header of the Clair3 output below.
I have also included the Spectre log with the SNV option enabled (the cmdline "contig" excluded), hope the info is a bit helpful. spectre_cnv.PAO89685_90Gbp.log
Kind regards,
Serge