ALLHiC icon indicating copy to clipboard operation
ALLHiC copied to clipboard

Published 20 hours ago • verified publisher icon tangerzhang

ALLHiC_partition ERROR Cannot open bamfile `sample.clean.bam` (gzip: invalid header)

Open chenji333 opened this issue 2 years ago • 4 comments

Hi, I am getting an error while running the ALLHiC_partition step. I skipped the Prune process. My command is ALLHiC_partition -b sample.clean.bam -r genome.polish.fasta -e AAGCTT -k 16 The running content is as follows:

Extract function: calculate an empirical distribution of Hi-C link size based on intra-contig links CMD: allhic extract sample.clean.bam female_genome.nextpolish.fasta --RE AAGCTT 10:30:08 writeRE | NOTICE RE counts in 1190 contigs (total: 597027, avg 1 per 3326 bp) written to sample.clean.counts_AAGCTT.txt 10:30:08 extractContigLinks | NOTICE Parse bamfile sample.clean.bam 10:30:08 extractContigLinks | ERROR Cannot open bamfile sample.clean.bam (gzip: invalid header) Partition contigs based on prunning bam file CMD: allhic partition sample.clean.counts_AAGCTT.txt sample.clean.pairs.txt 16 --minREs 25 10:30:09 ReadCSVLines | NOTICE Parse csvfile sample.clean.counts_AAGCTT.txt 10:30:09 readRE | NOTICE Loaded 1190 contig RE lengths for normalization from sample.clean.counts_AAGCTT.txt 10:30:09 skipContigsWithFewREs | NOTICE skipContigsWithFewREs with MinREs = 25 (RE = AAGCTT) Contig #3 (ctg000830_np12) has 21 RE sites -> MARKED SHORT Contig #4 (ctg000226_np12) has 11 RE sites -> MARKED SHORT Contig #10 (ctg000712_np12) has 15 RE sites -> MARKED SHORT Contig #18 (ctg000637_np12) has 22 RE sites -> MARKED SHORT Contig #21 (ctg000988_np12) has 19 RE sites -> MARKED SHORT Contig #39 (ctg000055_np12) has 10 RE sites -> MARKED SHORT Contig #47 (ctg001161_np12) has 19 RE sites -> MARKED SHORT Contig #65 (ctg000056_np12) has 4 RE sites -> MARKED SHORT Contig #68 (ctg000191_np12) has 3 RE sites -> MARKED SHORT Contig #82 (ctg000748_np12) has 21 RE sites -> MARKED SHORT Contig #85 (ctg000119_np12) has 21 RE sites -> MARKED SHORT Contig #98 (ctg000700_np12) has 15 RE sites -> MARKED SHORT Contig #102 (ctg000693_np12) has 20 RE sites -> MARKED SHORT Contig #103 (ctg000959_np12) has 24 RE sites -> MARKED SHORT Contig #104 (ctg000993_np12) has 16 RE sites -> MARKED SHORT Contig #108 (ctg000141_np12) has 23 RE sites -> MARKED SHORT Contig #123 (ctg000728_np12) has 13 RE sites -> MARKED SHORT Contig #127 (ctg000123_np12) has 4 RE sites -> MARKED SHORT Contig #128 (ctg000960_np12) has 12 RE sites -> MARKED SHORT Contig #146 (ctg001142_np12) has 22 RE sites -> MARKED SHORT Contig #151 (ctg000433_np12) has 24 RE sites -> MARKED SHORT Contig #161 (ctg000779_np12) has 12 RE sites -> MARKED SHORT Contig #168 (ctg000248_np12) has 24 RE sites -> MARKED SHORT Contig #172 (ctg000234_np12) has 7 RE sites -> MARKED SHORT Contig #173 (ctg000274_np12) has 17 RE sites -> MARKED SHORT Contig #174 (ctg001065_np12) has 5 RE sites -> MARKED SHORT Contig #176 (ctg001162_np12) has 20 RE sites -> MARKED SHORT Contig #177 (ctg000402_np12) has 21 RE sites -> MARKED SHORT Contig #188 (ctg000590_np12) has 10 RE sites -> MARKED SHORT Contig #193 (ctg000250_np12) has 20 RE sites -> MARKED SHORT Contig #219 (ctg000607_np12) has 23 RE sites -> MARKED SHORT Contig #222 (ctg000049_np12) has 24 RE sites -> MARKED SHORT Contig #228 (ctg000870_np12) has 18 RE sites -> MARKED SHORT Contig #239 (ctg001105_np12) has 18 RE sites -> MARKED SHORT Contig #249 (ctg000685_np12) has 1 RE sites -> MARKED SHORT Contig #260 (ctg000505_np12) has 16 RE sites -> MARKED SHORT Contig #273 (ctg000006_np12) has 24 RE sites -> MARKED SHORT Contig #287 (ctg000005_np12) has 1 RE sites -> MARKED SHORT Contig #289 (ctg000786_np12) has 16 RE sites -> MARKED SHORT Contig #294 (ctg000300_np12) has 21 RE sites -> MARKED SHORT Contig #298 (ctg000979_np12) has 14 RE sites -> MARKED SHORT Contig #303 (ctg000407_np12) has 21 RE sites -> MARKED SHORT Contig #308 (ctg001095_np12) has 19 RE sites -> MARKED SHORT Contig #340 (ctg000731_np12) has 11 RE sites -> MARKED SHORT Contig #367 (ctg000658_np12) has 1 RE sites -> MARKED SHORT Contig #377 (ctg000356_np12) has 22 RE sites -> MARKED SHORT Contig #383 (ctg000124_np12) has 1 RE sites -> MARKED SHORT Contig #387 (ctg000128_np12) has 13 RE sites -> MARKED SHORT Contig #391 (ctg000875_np12) has 13 RE sites -> MARKED SHORT Contig #404 (ctg000130_np12) has 14 RE sites -> MARKED SHORT Contig #424 (ctg001006_np12) has 23 RE sites -> MARKED SHORT Contig #452 (ctg000671_np12) has 23 RE sites -> MARKED SHORT Contig #457 (ctg000201_np12) has 9 RE sites -> MARKED SHORT Contig #474 (ctg000861_np12) has 21 RE sites -> MARKED SHORT Contig #480 (ctg000054_np12) has 17 RE sites -> MARKED SHORT Contig #485 (ctg000219_np12) has 23 RE sites -> MARKED SHORT Contig #500 (ctg000625_np12) has 17 RE sites -> MARKED SHORT Contig #502 (ctg000608_np12) has 11 RE sites -> MARKED SHORT Contig #516 (ctg000008_np12) has 20 RE sites -> MARKED SHORT Contig #523 (ctg000486_np12) has 24 RE sites -> MARKED SHORT Contig #526 (ctg001128_np12) has 18 RE sites -> MARKED SHORT Contig #527 (ctg000527_np12) has 10 RE sites -> MARKED SHORT Contig #556 (ctg000233_np12) has 18 RE sites -> MARKED SHORT Contig #569 (ctg000322_np12) has 17 RE sites -> MARKED SHORT Contig #592 (ctg001108_np12) has 15 RE sites -> MARKED SHORT Contig #594 (ctg000050_np12) has 24 RE sites -> MARKED SHORT Contig #595 (ctg000138_np12) has 4 RE sites -> MARKED SHORT Contig #603 (ctg000669_np12) has 21 RE sites -> MARKED SHORT Contig #614 (ctg000406_np12) has 13 RE sites -> MARKED SHORT Contig #620 (ctg001038_np12) has 11 RE sites -> MARKED SHORT Contig #647 (ctg000450_np12) has 9 RE sites -> MARKED SHORT Contig #677 (ctg000730_np12) has 19 RE sites -> MARKED SHORT Contig #687 (ctg000022_np12) has 7 RE sites -> MARKED SHORT Contig #690 (ctg000083_np12) has 23 RE sites -> MARKED SHORT Contig #696 (ctg000882_np12) has 10 RE sites -> MARKED SHORT Contig #701 (ctg000436_np12) has 15 RE sites -> MARKED SHORT Contig #702 (ctg000824_np12) has 17 RE sites -> MARKED SHORT Contig #719 (ctg000241_np12) has 21 RE sites -> MARKED SHORT Contig #752 (ctg001013_np12) has 17 RE sites -> MARKED SHORT Contig #761 (ctg001011_np12) has 21 RE sites -> MARKED SHORT Contig #765 (ctg000937_np12) has 12 RE sites -> MARKED SHORT Contig #779 (ctg000699_np12) has 16 RE sites -> MARKED SHORT Contig #787 (ctg000789_np12) has 1 RE sites -> MARKED SHORT Contig #794 (ctg000946_np12) has 22 RE sites -> MARKED SHORT Contig #805 (ctg000341_np12) has 9 RE sites -> MARKED SHORT Contig #817 (ctg000214_np12) has 24 RE sites -> MARKED SHORT Contig #824 (ctg001023_np12) has 20 RE sites -> MARKED SHORT Contig #846 (ctg000587_np12) has 16 RE sites -> MARKED SHORT Contig #859 (ctg000038_np12) has 17 RE sites -> MARKED SHORT Contig #880 (ctg000090_np12) has 20 RE sites -> MARKED SHORT Contig #881 (ctg001087_np12) has 5 RE sites -> MARKED SHORT Contig #882 (ctg000210_np12) has 19 RE sites -> MARKED SHORT Contig #884 (ctg000091_np12) has 3 RE sites -> MARKED SHORT Contig #890 (ctg000514_np12) has 22 RE sites -> MARKED SHORT Contig #908 (ctg000812_np12) has 19 RE sites -> MARKED SHORT Contig #912 (ctg000397_np12) has 10 RE sites -> MARKED SHORT Contig #919 (ctg000956_np12) has 3 RE sites -> MARKED SHORT Contig #924 (ctg001155_np12) has 19 RE sites -> MARKED SHORT Contig #932 (ctg000951_np12) has 17 RE sites -> MARKED SHORT Contig #937 (ctg000075_np12) has 12 RE sites -> MARKED SHORT Contig #940 (ctg000352_np12) has 22 RE sites -> MARKED SHORT Contig #946 (ctg001172_np12) has 22 RE sites -> MARKED SHORT Contig #951 (ctg000335_np12) has 4 RE sites -> MARKED SHORT Contig #986 (ctg000044_np12) has 22 RE sites -> MARKED SHORT Contig #989 (ctg000023_np12) has 23 RE sites -> MARKED SHORT Contig #1005 (ctg000027_np12) has 6 RE sites -> MARKED SHORT Contig #1011 (ctg000752_np12) has 8 RE sites -> MARKED SHORT Contig #1015 (ctg000507_np12) has 18 RE sites -> MARKED SHORT Contig #1016 (ctg001077_np12) has 24 RE sites -> MARKED SHORT Contig #1037 (ctg000409_np12) has 18 RE sites -> MARKED SHORT Contig #1075 (ctg000085_np12) has 21 RE sites -> MARKED SHORT Contig #1085 (ctg001139_np12) has 20 RE sites -> MARKED SHORT Contig #1099 (ctg001121_np12) has 16 RE sites -> MARKED SHORT Contig #1109 (ctg000068_np12) has 1 RE sites -> MARKED SHORT Contig #1143 (ctg000142_np12) has 13 RE sites -> MARKED SHORT Contig #1153 (ctg000564_np12) has 21 RE sites -> MARKED SHORT Contig #1163 (ctg001185_np12) has 17 RE sites -> MARKED SHORT Contig #1164 (ctg001073_np12) has 24 RE sites -> MARKED SHORT 10:30:09 skipContigsWithFewREs | NOTICE Marked 118 contigs (avg 15.7 RE sites, len 191174) since they contain too few REs (MinREs = 25) 10:30:09 ReadCSVLines | NOTICE Parse csvfile sample.clean.pairs.txt 10:30:09 mustOpen | CRITIC open sample.clean.pairs.txt: no such file or directory

I think the problem might be "Cannot open bamfile", but I don't know how to run it.

chenji333 avatar Apr 24 '22 03:04 chenji333

Hi， Error of ERROR Cannot open bamfile sample.clean.bam (gzip: invalid header) occurred. How did you generate the sample.clean.bam file? And what is the version of samtools your used? Please provide more detail information.

wangyibin avatar Apr 24 '22 05:04 wangyibin

I created environment for allhic using "conda create -y -n allhic python=3.7 samtools bedtools matplotlib" command. The version information of samtools is: Program: samtools (Tools for alignments in the SAM format) Version: 1.15.1 (using htslib 1.15.1)

I use the sample.clean.bam generated by the command like this: nohup filterBAM_forHiC.pl sample.bwa_aln.REduced.paired_only.bam sample.clean.sam > clean.log & nohup samtools view -bt genome.polish.fasta.fai sample.clean.sam > sample.clean.bam 2>&1 &

My initial run command was: samtools faidx genome.polish.fasta bwa index -a bwtsw genome.polish.fasta

bash.sh bwa aln -t 24 genome.polish.fasta Clean_1.fq.gz > reads_R1.sai bwa aln -t 24 genome.polish.fasta Clean_2.fq.gz > reads_R2.sai bwa sampe genome.polish.fasta reads_R1.sai reads_R2.sai Clean_1.fq.gz Clean_2.fq.gz > sample.bwa_aln.sam nohup sh bash.sh & nohup PreprocessSAMs.pl sample.bwa_aln.sam genome.polish.fasta HindIII > PreprocessSAMs.log &

chenji333 avatar Apr 24 '22 07:04 chenji333

Hi, You should run command with nohup by this way:

nohup samtools view -bt genome.polish.fasta.fai sample.clean.sam -o sample.clean.bam &

It is an error to output the result to standard output when using the nohup command.

wangyibin avatar Apr 24 '22 07:04 wangyibin

I just changed 'nohup' and it worked. Thank you very much for solving my problem.

chenji333 avatar Apr 24 '22 08:04 chenji333