conterminator
conterminator copied to clipboard
Published
20 hours ago •
steineggerlab
steineggerlab
Invalid pointer during calculation of alignments
Version: commit 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7
Command:
./conterminator/build/bin/conterminator protein GTDB97.faa taxonomy/mapping_new conterminator_phylum_level conterminator_tmp --threads 64 --ncbi-tax-dump taxonomy --kingdoms 3,9,22,28,71,85,104,113,188,435,558,600,661,798,948,2112,2179,2185,2197,2222,2231,2290,2346,2367,2457,2526,2599,2662,2804,2844,2936,2947,3026,3093,3196,3236,3259,3295,3332,3478,3486,3519,3588,3721,3764,3806,3852,3869,3971,4049,4123,4138,4295,4380,4393,4426,4438,4480,4574,4637,4747,4807,4962,5058,5211,5295,5301,5341,5428,5455,5610,5785,5850,5876,6053,6077,6352,6994,7152,7497,7678,7997,8069,8163,8265,8415,8508,9033,9459,9518,9602,9620,9659,9816,9982,10092,10112,10373,10409,10624,10701,10941,10980,11478,12122,12426,12439,12474,12788,12916,13990,14290,14512,14627,14692,14712,14955,15712,16418,16488,17029,17211,17233,17428,17878,17918,18303,18551,19564,19757,19952,20114,20624,21463,22205,22539,22684,23843,24111,24497,26693,27254,29002,29170,30513,31392,32011,33650,34643,34865,35143
Output:
Tmp conterminator_tmp folder does not exist or is not a directory.
Create dir conterminator_tmp
protein GTDB97.faa taxonomy/mapping_new conterminator_phylum_level conterminator_tmp --threads 64 --ncbi-tax-dump taxonomy --kingdoms 3,9,22,28,71,85,104,113,188,435,558,600,661,798,948,2112,2179,2185,2197,2222,2231,2290,2346,2367,2457,2526,2599,2662,2804,2844,2936,2947,3026,3093,3196,3236,3259,3295,3332,3478,3486,3519,3588,3721,3764,3806,3852,3869,3971,4049,4123,4138,4295,4380,4393,4426,4438,4480,4574,4637,4747,4807,4962,5058,5211,5295,5301,5341,5428,5455,5610,5785,5850,5876,6053,6077,6352,6994,7152,7497,7678,7997,8069,8163,8265,8415,8508,9033,9459,9518,9602,9620,9659,9816,9982,10092,10112,10373,10409,10624,10701,10941,10980,11478,12122,12426,12439,12474,12788,12916,13990,14290,14512,14627,14692,14712,14955,15712,16418,16488,17029,17211,17233,17428,17878,17918,18303,18551,19564,19757,19952,20114,20624,21463,22205,22539,22684,23843,24111,24497,26693,27254,29002,29170,30513,31392,32011,33650,34643,34865,35143
MMseqs Version: 3eabfaff83bb77eac5ef342e8905cc4f7d378cb7
Substitution matrix nucl:nucleotide.out,aa:blosum62.out
Add backtrace false
Alignment mode 3
Allow wrapped scoring false
E-value threshold 0.001
Seq. id. threshold 0.95
Min. alignment length 0
Seq. id. mode 0
Alternative alignments 0
Coverage threshold 0.95
Coverage mode 0
Max sequence length 65535
Compositional bias 1
Realign hits false
Max reject 2147483647
Max accept 2147483647
Include identical seq. id. false
Preload mode 0
Pseudo count a 1
Pseudo count b 1.5
Score bias 0
Gap open cost 11
Gap extension cost 1
Threads 64
Compressed 0
Verbosity 3
Seed substitution matrix nucl:nucleotide.out,aa:VTML80.out
Sensitivity 4
K-mer size 0
K-score 2147483647
Alphabet size 21
Split database 0
Split mode 2
Split memory limit 0
Diagonal scoring true
Exact k-mer matching 0
Mask residues 0
Mask lower case residues 0
Minimum diagonal score 15
Spaced k-mers 1
Spaced k-mer pattern
Local temporary path
Rescore mode 0
Remove hits by seq. id. and coverage false
Sort results 0
Mask profile 1
Profile e-value threshold 0.001
Use global sequence weighting false
Allow deletions false
Filter MSA 1
Maximum seq. id. threshold 0.9
Minimum seq. id. 0
Minimum score per column -20
Minimum coverage 0
Select N most diverse seqs 1000
Omit consensus false
Min codons in orf 30
Max codons in length 32734
Max orf gaps 2147483647
Contig start mode 2
Contig end mode 2
Orf start mode 1
Forward frames 1,2,3
Reverse frames 1,2,3
Translation table 1
Translate orf 0
Use all table starts false
Offset of numeric ids 0
Create lookup 0
Add orf stop false
createdb GTDB97.faa conterminator_tmp/6937418719709452993/sequencedb
Converting sequences
[=================================================================================================== 1 Mio. sequences processed
=================================================================================================== 2 Mio. sequences processed
=================================================================================================== 3 Mio. sequences processed
=================================================================================================== 4 Mio. sequences processed
=================================================================================================== 5 Mio. sequences processed
=================================================================================================== 6 Mio. sequences processed
=================================================================================================== 7 Mio. sequences processed
=================================================================================================== 8 Mio. sequences processed
=================================================================================================== 9 Mio. sequences processed
=================================================================================================== 10 Mio. sequences processed
=================================================================================================== 11 Mio. sequences processed
=================================================================================================== 12 Mio. sequences processed
=================================================================================================== 13 Mio. sequences processed
=================================================================================================== 14 Mio. sequences processed
=================================================================================================== 15 Mio. sequences processed
=================================================================================================== 16 Mio. sequences processed
=================================================================================================== 17 Mio. sequences processed
=================================================================================================== 18 Mio. sequences processed
=================================================================================================== 19 Mio. sequences processed
=================================================================================================== 20 Mio. sequences processed
=================================================================================================== 21 Mio. sequences processed
=================================================================================================== 22 Mio. sequences processed
=================================================================================================== 23 Mio. sequences processed
=================================================================================================== 24 Mio. sequences processed
=================================================================================================== 25 Mio. sequences processed
=================================================================================================== 26 Mio. sequences processed
=================================================================================================== 27 Mio. sequences processed
=================================================================================================== 28 Mio. sequences processed
=================================================================================================== 29 Mio. sequences processed
=================================================================================================== 30 Mio. sequences processed
=================================================================================================== 31 Mio. sequences processed
=================================================================================================== 32 Mio. sequences processed
=================================================================================================== 33 Mio. sequences processed
=================================================================================================== 34 Mio. sequences processed
=================================================================================================== 35 Mio. sequences processed
=================================================================================================== 36 Mio. sequences processed
=================================================================================================== 37 Mio. sequences processed
=================================================================================================== 38 Mio. sequences processed
=================================================================================================== 39 Mio. sequences processed
=================================================================================================== 40 Mio. sequences processed
=================================================================================================== 41 Mio. sequences processed
=================================================================================================== 42 Mio. sequences processed
=================================================================================================== 43 Mio. sequences processed
=================================================================================================== 44 Mio. sequences processed
=================================================================================================== 45 Mio. sequences processed
=================================================================================================== 46 Mio. sequences processed
=================================================================================================== 47 Mio. sequences processed
=================================================================================================== 48 Mio. sequences processed
=================================================================================================== 49 Mio. sequences processed
=================================================================================================== 50 Mio. sequences processed
=================================================================================================== 51 Mio. sequences processed
=================================================================================================== 52 Mio. sequences processed
=================================================================================================== 53 Mio. sequences processed
=================================================================================================== 54 Mio. sequences processed
=================================================================================================== 55 Mio. sequences processed
=================================================================================================== 56 Mio. sequences processed
=================================================================================================== 57 Mio. sequences processed
=================================================================================================== 58 Mio. sequences processed
=================================================================================================== 59 Mio. sequences processed
=================================================================================================== 60 Mio. sequences processed
=================================================================================================== 61 Mio. sequences processed
=================================================================================================== 62 Mio. sequences processed
=================================================================================================== 63 Mio. sequences processed
=================================================================================================== 64 Mio. sequences processed
=================================================================================================== 65 Mio. sequences processed
=================================================================================================== 66 Mio. sequences processed
=================================================================================================== 67 Mio. sequences processed
=================================================================================================== 68 Mio. sequences processed
=================================================================================================== 69 Mio. sequences processed
=================================================================================================== 70 Mio. sequences processed
=================================================================================================== 71 Mio. sequences processed
=================================================================================================== 72 Mio. sequences processed
=================================================================================================== 73 Mio. sequences processed
===========================================================================
Time for merging to sequencedb_h: 0h 0m 39s 357ms
Time for merging to sequencedb: 0h 1m 3s 797ms
Database type: Aminoacid
Time for merging to sequencedb.lookup: 0h 0m 0s 3ms
Time for processing: 0h 4m 39s 300ms
Tmp conterminator_tmp/6937418719709452993/createtaxdb folder does not exist or is not a directory.
Download taxdump.tar.gz
2020-05-05 13:45:37 URL:https://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz [51998442/51998442] -> "-" [1]
Database created
Remove temporary files
conterminator_tmp/6937418719709452993/createtaxdb/createindex.sh: line 58: [: : integer expression expected
Tmp conterminator_tmp/6937418719709452993/linclust folder does not exist or is not a directory.
kmermatcher conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref --sub-mat nucl:nucleotide.out,aa:blosum62.out --alph-size 21 --min-seq-id 0.95 --kmer-per-seq 100 --spaced-kmer-mode 1 --kmer-per-seq-scale 0 --adjust-kmer-len 0 --mask 0 --mask-lower-case 0 --cov-mode 0 -k 0 -c 0.95 --max-seq-len 65535 --hash-shift 67 --split-memory-limit 0 --include-only-extendable 0 --ignore-multi-kmer 0 --threads 64 --compressed 0 -v 3
kmermatcher conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref --sub-mat nucl:nucleotide.out,aa:blosum62.out --alph-size 21 --min-seq-id 0.95 --kmer-per-seq 100 --spaced-kmer-mode 1 --kmer-per-seq-scale 0 --adjust-kmer-len 0 --mask 0 --mask-lower-case 0 --cov-mode 0 -k 0 -c 0.95 --max-seq-len 65535 --hash-shift 67 --split-memory-limit 0 --include-only-extendable 0 --ignore-multi-kmer 0 --threads 64 --compressed 0 -v 3
Database size: 73751620 type: Aminoacid
Reduced amino acid alphabet: (A S T) (C) (D B N) (E Q Z) (F Y) (G) (H) (I V) (K R) (L J M) (P) (W) (X)
Not enough memory to process at once need to split
[=================================================================] 73.75M 43s 227ms
Process file into 2 parts
Generate k-mers list for 1 split
[=================================================================] 73.75M 57s 821ms
Sort kmer 0h 2m 44s 971ms
Sort by rep. sequence 0h 0m 36s 775ms
Generate k-mers list for 2 split
[=================================================================] 73.75M 59s 874ms
Sort kmer 0h 0m 21s 36ms
Sort by rep. sequence 0h 0m 3s 483ms
Merge splits ... Time for fill: 0h 1m 32s 459ms
Time for merging to pref: 0h 0m 22s 166ms
Time for processing: 0h 11m 40s 553ms
rescorediagonal conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore1 --sub-mat nucl:nucleotide.out,aa:blosum62.out --rescore-mode 0 --wrapped-scoring 0 --filter-hits 0 -e 0.001 -c 0.95 -a 0 --cov-mode 0 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --add-self-matches 0 --sort-results 0 --db-load-mode 0 --threads 64 --compressed 0 -v 3
[=================================================================] 73.75M 30s 492ms
Time for merging to pref_rescore1: 0h 0m 30s 761ms
Time for processing: 0h 1m 32s 935ms
clust conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore1 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pre_clust --cluster-mode 0 --max-iterations 1000 --similarity-type 2 --threads 64 --compressed 0 -v 3
Clustering mode: Set Cover
[=================================================================] 1.00M 0s 65ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 201ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 198ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 194ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 194ms
[=================================================================] 1.00M 0s 192ms
[=================================================================] 1.00M 0s 191ms
[=================================================================] 1.00M 0s 188ms
[=================================================================] 1.00M 0s 186ms
[=================================================================] 1.00M 0s 183ms
[=================================================================] 1.00M 0s 178ms
[=================================================================] 1.00M 0s 174ms
[=================================================================] 1.00M 0s 169ms
[=================================================================] 1.00M 0s 155ms
[=================================================================] 1.00M 0s 131ms
[=================================================================] 751.62K 0s 94ms
Sort entries
Find missing connections
Found 6021788 new connections.
Reconstruct initial order
[=================================================================] 1.00M 0s 194ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 197ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 196ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 195ms
[=================================================================] 1.00M 0s 194ms
[=================================================================] 1.00M 0s 194ms
[=================================================================] 1.00M 0s 193ms
[=================================================================] 1.00M 0s 192ms
[=================================================================] 1.00M 0s 190ms
[=================================================================] 1.00M 0s 188ms
[=================================================================] 1.00M 0s 185ms
[=================================================================] 1.00M 0s 183ms
[=================================================================] 1.00M 0s 177ms
[=================================================================] 1.00M 0s 173ms
[=================================================================] 1.00M 0s 169ms
[=================================================================] 1.00M 0s 155ms
[=================================================================] 1.00M 0s 131ms
[=================================================================] 751.62K 0s 95ms
Add missing connections
[=================================================================] 73.75M 3s 71ms
Time for read in: 0h 0m 55s 134ms
Total time: 0h 1m 10s 664ms
Size of the sequence database: 73751620
Size of the alignment database: 73751620
Number of clusters: 69196986
Writing results 0h 0m 25s 185ms
Time for merging to pre_clust: 0h 0m 20s 696ms
Time for processing: 0h 2m 19s 837ms
createsubdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy conterminator_tmp/6937418719709452993/sequencedb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy -v 3 --subdb-mode 1
Time for merging to input_step_redundancy: 0h 0m 9s 486ms
Time for processing: 0h 0m 25s 257ms
createsubdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter1 -v 3 --subdb-mode 1
Time for merging to pref_filter1: 0h 0m 9s 362ms
Time for processing: 0h 0m 25s 757ms
filterdb conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter1 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter2 --filter-file conterminator_tmp/6937418719709452993/linclust/7399153080635485316/order_redundancy
Filtering using file(s)
[=================================================================] 69.20M 18s 92ms
Time for merging to pref_filter2: 0h 0m 30s 266ms
Time for processing: 0h 1m 5s 686ms
rescorediagonal conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_filter2 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore2 --sub-mat nucl:nucleotide.out,aa:blosum62.out --rescore-mode 1 --wrapped-scoring 0 --filter-hits 1 -e 0.001 -c 0.95 -a 0 --cov-mode 0 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --add-self-matches 0 --sort-results 0 --db-load-mode 0 --threads 64 --compressed 0 -v 3
[=================================================================] 69.20M 29s 670ms
Time for merging to pref_rescore2: 0h 0m 29s 593ms
Time for processing: 0h 1m 19s 75ms
align conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/input_step_redundancy conterminator_tmp/6937418719709452993/linclust/7399153080635485316/pref_rescore2 conterminator_tmp/6937418719709452993/linclust/7399153080635485316/aln --sub-mat nucl:nucleotide.out,aa:blosum62.out -a 0 --alignment-mode 3 --wrapped-scoring 0 -e 0.001 --min-seq-id 0.95 --min-aln-len 0 --seq-id-mode 0 --alt-ali 0 -c 0.95 --cov-mode 0 --max-seq-len 65535 --comp-bias-corr 1 --realign 0 --max-rejected 2147483647 --max-accept 2147483647 --add-self-matches 0 --db-load-mode 0 --pca 1 --pcb 1.5 --score-bias 0 --gap-open 11 --gap-extend 1 --threads 64 --compressed 0 -v 3
Compute score, coverage and sequence identity
Query database size: 69196986 type: Aminoacid
Target database size: 69196986 type: Aminoacid
Calculation of alignments
*** Error in `/global/cscratch1/sd/acamargo/GTDB/./conterminator/build/bin/conterminator': munmap_chunk(): invalid pointer: 0x00002ab2d008fe80 ***
conterminator_tmp/6937418719709452993/linclust/7399153080635485316/linclust.sh: line 75: 27861 Aborted $RUNNER "$MMSEQS" "${ALIGN_MODULE}" "$INPUT" "$INPUT" "$RESULTDB" "${TMP_PATH}/aln" ${ALIGNMENT_PAR}
[=Error: Alignment step died
Error: kmermatcher step died
srun: error: nid00369: task 0: Exited with exit code 1
srun: Terminating job step 30385822.0
