parsnp icon indicating copy to clipboard operation
parsnp copied to clipboard
verified publisher icon marbl

Error in run of 732 genomes

Open naifahim opened this issue 8 months ago • 21 comments

Hlw,

I am facing a few problems:

  1. I have 732 genomes and 10 of my own with a reference. All of these are assembled. When i am running parsnp, it’s showing failed. First it gives me a tree but no value. Other times failed except one time when it gave me a tree of only 20-25 genomes.

  2. Then i went through quast Qc, and saw rhat qc are not good enough in few genomes. Like few are not good at total legnth, few for contigs, few for N50, and few for N based pair. So i have gone through few tactics. I have checked with only good QC of 20-30 genomes and it gives me tree. Im second time, i have gone through only bad Qc genome of contigs, N50,L50 of 20-30 genomes, it also gives tree.

However, when i have run a few gemomes of bad N 100 base, it does not give me tree.

So i decided to exclude these. After excluding, i have run around 680 genomes from 732,,but it didnot give me tree. Direct terminated.

So how can i resolve these issue. What coulde be the error.

Let me know please

naifahim avatar Jun 21 '25 18:06 naifahim

Hi @naifahim,

Could you please run your command with --verbose flag and then share both the command and the output? Thank you!

bkille avatar Jun 22 '25 01:06 bkille

Thank you for your response. Before doing it, i have a question. Is this command or tools related to storage of the laptop? I have 40gb space left in the folder where i am using linux.

The total genomes number are 732.

Please let me know. Because when i am using it, linux is getting shut down.

On Sun, Jun 22, 2025, 7:01 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-2993850814

Hi @naifahim https://github.com/naifahim,

Could you please run your command with --verbose flag and then share both the command and the output? Thank you!

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-2993850814, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKCZYTKRWIESUE64KNL3EX555AVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJTHA2TAOBRGQ . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 22 '25 03:06 naifahim

(parsnp-new) @.***:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ parsnp -c -p 8 -r reference.fna -d a -a c -o result --verbose 09:11:45 - INFO - |--Parsnp 2.1.4--|

09:11:47 - INFO -

SETTINGS: |-refgenome: reference.fna |-genomes: a/GCA_000285655.3_EC958.v1_genomic.fna a/GCA_002224705.1_ASM222470v1_genomic.fna ...681 more file(s)... a/MTR_ET11.fasta a/MTR_ET12.fasta |-aligner: muscle |-outdir: result |-OS: Linux |-threads: 8

09:11:47 - WARNING - You have asked to use more threads than you have available on your machine. This may lead to serious performance degredation with RAxML. 09:11:47 - INFO - <<Parsnp started>> 09:11:47 - INFO - No genbank file provided for reference annotations, skipping.. 09:15:04 - DEBUG - Writing .ini file 09:15:04 - INFO - Setting the partition size to 52 09:15:04 - INFO - Running partitions... 09:15:05 - INFO - 0%| | 0/14 [00:00<?, ?it/s] 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000004-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000006-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000000-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000001-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000007-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000005-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000002-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000003-out/parsnpAligner.ini Terminated (parsnp-new) @.*** :/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$

Here, it is terminated automatically

Message ID: @.***>

naifahim avatar Jun 22 '25 03:06 naifahim

(base) @.***:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ parsnp -c -r reference.fna -d a -a c -o result --verbose |--Parsnp v1.2--| For detailed documentation please see --> http://harvest.readthedocs.org/en/latest

SETTINGS: |-refgenome: reference.fna |-aligner: libMUSCLE |-seqdir: a |-outdir: result |-OS: Linux |-threads: 32

<<Parsnp started>>

-->Reading Genome (asm, fasta) files from a.. |->[OK] -->Reading Genbank file(s) for reference (.gbk) .. |->[WARNING]: no genbank file provided for reference annotations, skipping.. RECRUITED GENOMES:

-->Running Parsnp multi-MUM search and libMUSCLE aligner.. ERROR The following command failed:

/tmp/_MEIRRFUr3/parsnp result/parsnpAligner.ini Please veryify input data and restart Parsnp. If the problem persists please contact the Parsnp development team. ERROR

(base) @.***:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$

On Sun, Jun 22, 2025 at 9:18 AM Naeem Ahammed Ibrahim Fahim < @.***> wrote:

(parsnp-new) @.***:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ parsnp -c -p 8 -r reference.fna -d a -a c -o result --verbose 09:11:45 - INFO - |--Parsnp 2.1.4--|

09:11:47 - INFO -

SETTINGS: |-refgenome: reference.fna |-genomes: a/GCA_000285655.3_EC958.v1_genomic.fna a/GCA_002224705.1_ASM222470v1_genomic.fna ...681 more file(s)... a/MTR_ET11.fasta a/MTR_ET12.fasta |-aligner: muscle |-outdir: result |-OS: Linux |-threads: 8

09:11:47 - WARNING - You have asked to use more threads than you have available on your machine. This may lead to serious performance degredation with RAxML. 09:11:47 - INFO - <<Parsnp started>> 09:11:47 - INFO - No genbank file provided for reference annotations, skipping.. 09:15:04 - DEBUG - Writing .ini file 09:15:04 - INFO - Setting the partition size to 52 09:15:04 - INFO - Running partitions... 09:15:05 - INFO - 0%| | 0/14 [00:00<?, ?it/s] 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000004-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000006-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000000-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000001-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000007-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000005-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000002-out/parsnpAligner.ini 09:15:05 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000003-out/parsnpAligner.ini Terminated (parsnp-new) @.*** :/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$

Here, it is terminated automatically

Message ID: @.***>

naifahim avatar Jun 22 '25 03:06 naifahim

Hi @naifahim

40gb of storage should be enough for parsnp. May I ask how much RAM you have on your machine? Also, you are requesting 8 threads, which according to the warning in the output is more than the number available in your machine. I'd suggest running with fewer threads.

Also, it appears you still have an old version of parsnp (your most recent comment is from parsnp 1.2)

bkille avatar Jun 22 '25 03:06 bkille

(base) fahim@DESKTOP-AJGON69:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ conda activate parsnp-new (parsnp-new) fahim@DESKTOP-AJGON69:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ parsnp -c -r reference.fna -d a -a c -o result --verbose 09:44:16 - INFO - |--Parsnp 2.1.4--|

09:44:18 - INFO -

SETTINGS: |-refgenome: reference.fna |-genomes: a/GCA_000285655.3_EC958.v1_genomic.fna a/GCA_002224705.1_ASM222470v1_genomic.fna ...681 more file(s)... a/MTR_ET11.fasta a/MTR_ET12.fasta |-aligner: muscle |-outdir: result |-OS: Linux |-threads: 1

09:44:18 - INFO - <<Parsnp started>> 09:44:18 - INFO - No genbank file provided for reference annotations, skipping.. 09:47:34 - DEBUG - Writing .ini file 09:47:34 - INFO - Setting the partition size to 52 09:47:35 - INFO - Running partitions... 09:47:35 - INFO - 0%| | 0/14 [00:00<?, ?it/s] 09:47:35 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000000-out/parsnpAligner.ini 09:48:09 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000000-out/log/parsnp-aligner.out 09:48:09 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000000-out/log/parsnp-aligner.err 09:48:09 - DEBUG - 09:48:09 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:48:09 - ERROR - Partition 0000000000 failed... 09:48:09 - INFO - 7%|7 | 1/14 [00:33<07:19, 33.82s/it] 09:48:09 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000001-out/parsnpAligner.ini 09:48:41 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000001-out/log/parsnp-aligner.out 09:48:41 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000001-out/log/parsnp-aligner.err 09:48:41 - DEBUG - 09:48:41 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:48:41 - ERROR - Partition 0000000001 failed... 09:48:41 - INFO - 14%|#4 | 2/14 [01:05<06:33, 32.80s/it] 09:48:41 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000002-out/parsnpAligner.ini 09:49:13 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000002-out/log/parsnp-aligner.out 09:49:13 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000002-out/log/parsnp-aligner.err 09:49:13 - DEBUG - 09:49:13 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:49:13 - ERROR - Partition 0000000002 failed... 09:49:13 - INFO - 21%|##1 | 3/14 [01:38<05:57, 32.53s/it] 09:49:13 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000003-out/parsnpAligner.ini 09:49:44 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000003-out/log/parsnp-aligner.out 09:49:44 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000003-out/log/parsnp-aligner.err 09:49:44 - DEBUG - 09:49:44 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:49:44 - ERROR - Partition 0000000003 failed... 09:49:44 - INFO - 29%|##8 | 4/14 [02:09<05:19, 31.97s/it] 09:49:44 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000004-out/parsnpAligner.ini 09:50:16 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000004-out/log/parsnp-aligner.out 09:50:16 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000004-out/log/parsnp-aligner.err 09:50:16 - DEBUG - 09:50:16 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:50:16 - ERROR - Partition 0000000004 failed... 09:50:16 - INFO - 36%|###5 | 5/14 [02:41<04:47, 31.97s/it] 09:50:16 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000005-out/parsnpAligner.ini 09:50:54 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000005-out/log/parsnp-aligner.out 09:50:54 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000005-out/log/parsnp-aligner.err 09:50:54 - DEBUG - 09:50:54 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:50:54 - ERROR - Partition 0000000005 failed... 09:50:54 - INFO - 43%|####2 | 6/14 [03:19<04:31, 33.98s/it] 09:50:54 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000006-out/parsnpAligner.ini 09:51:26 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000006-out/log/parsnp-aligner.out 09:51:26 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000006-out/log/parsnp-aligner.err 09:51:26 - DEBUG - 09:51:26 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:51:26 - ERROR - Partition 0000000006 failed... 09:51:26 - INFO - 50%|##### | 7/14 [03:50<03:51, 33.13s/it] 09:51:26 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000007-out/parsnpAligner.ini 09:51:56 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000007-out/log/parsnp-aligner.out 09:51:56 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000007-out/log/parsnp-aligner.err 09:51:56 - DEBUG - 09:51:56 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:51:56 - ERROR - Partition 0000000007 failed... 09:51:56 - INFO - 57%|#####7 | 8/14 [04:21<03:14, 32.33s/it] 09:51:56 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000008-out/parsnpAligner.ini 09:52:27 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000008-out/log/parsnp-aligner.out 09:52:27 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000008-out/log/parsnp-aligner.err 09:52:27 - DEBUG - 09:52:27 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:52:27 - ERROR - Partition 0000000008 failed... 09:52:27 - INFO - 64%|######4 | 9/14 [04:52<02:39, 31.90s/it] 09:52:27 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000009-out/parsnpAligner.ini 09:52:58 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000009-out/log/parsnp-aligner.out 09:52:58 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000009-out/log/parsnp-aligner.err 09:52:58 - DEBUG - 09:52:58 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:52:58 - ERROR - Partition 0000000009 failed... 09:52:58 - INFO - 71%|#######1 | 10/14 [05:23<02:06, 31.74s/it] 09:52:58 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000010-out/parsnpAligner.ini 09:53:29 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000010-out/log/parsnp-aligner.out 09:53:29 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000010-out/log/parsnp-aligner.err 09:53:29 - DEBUG - 09:53:29 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:53:29 - ERROR - Partition 0000000010 failed... 09:53:29 - INFO - 79%|#######8 | 11/14 [05:53<01:34, 31.38s/it] 09:53:29 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000011-out/parsnpAligner.ini 09:54:01 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000011-out/log/parsnp-aligner.out 09:54:01 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000011-out/log/parsnp-aligner.err 09:54:01 - DEBUG - 09:54:01 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:54:01 - ERROR - Partition 0000000011 failed... 09:54:01 - INFO - 86%|########5 | 12/14 [06:26<01:03, 31.63s/it] 09:54:01 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000012-out/parsnpAligner.ini 09:54:33 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000012-out/log/parsnp-aligner.out 09:54:33 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000012-out/log/parsnp-aligner.err 09:54:33 - DEBUG - 09:54:33 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:54:33 - ERROR - Partition 0000000012 failed... 09:54:33 - INFO - 93%|#########2| 13/14 [06:57<00:31, 31.55s/it] 09:54:33 - DEBUG - time /home/fahim/miniconda3/envs/parsnp-new/bin/bin/parsnp_core result/partition/chunk-0000000013-out/parsnpAligner.ini 09:54:39 - DEBUG - STDOUT was piped... please see output file: result/partition/chunk-0000000013-out/log/parsnp-aligner.out 09:54:39 - DEBUG - STDERR was piped... please see output file: result/partition/chunk-0000000013-out/log/parsnp-aligner.err 09:54:39 - DEBUG - 09:54:39 - CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering 09:54:39 - ERROR - Partition 0000000013 failed... 09:54:39 - INFO - 93%|#########2| 13/14 [07:04<00:32, 32.64s/it] 09:54:39 - INFO - Computing intersection of all partition LCBs... Traceback (most recent call last): File "/home/fahim/miniconda3/envs/parsnp-new/bin/parsnp", line 1593, in intersected_interval_dict = partition.get_intersected_intervals(chunk_to_intvervaldict) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fahim/miniconda3/envs/parsnp-new/bin/partition.py", line 552, in get_intersected_intervals first_chunk = list(chunk_to_invervaldict.keys())[0] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^ IndexError: list index out of range (parsnp-new) fahim@DESKTOP-AJGON69:/mnt/d/project/4/e.coli/2pro/parsnp/pubmed$ parsnp version 09:55:02 - INFO - |--Parsnp 2.1.4--|

naifahim avatar Jun 22 '25 04:06 naifahim

Thank you for your email. please have a look. i have run it in env of 2.1.4 version without thread.

I have 4gb RAM. is it a issue? it seems like the error in partitions?

I have attached my quast result of all genomes. could please check to be sure are all okay?

I have posted the message in github with error and attached the quast file in drive link.

https://drive.google.com/file/d/1g5cZil1D4hxiTr7iDeHvN05p9QiiiV2x/view?usp=drive_link

naifahim avatar Jun 22 '25 05:06 naifahim

Working on the same project (same genomes), I could generate a tree file, but the tree shows zero height in iTOL. I have attached the output folder in this Google Drive link - https://drive.google.com/drive/folders/1Cn8iXQ5cCiazo0gxXh9oR8Ke7LkwEw5t?usp=sharing

Here are some snippets from the terminal after running:

HEAD (parsnp-env) afsan@afsan-ubuntu:~$ parsnp -r /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/GCA_000005845.2_ASM584v2_genomic.fna -g /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/MG1655.gbk -d /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/ -o /home/afsan/Ecoli10st/Analysis/Parsnp -c -v -p 1 --fo 11:37:45 - INFO - |--Parsnp 2.1.4--|

11:37:45 - WARNING - Output directory /home/afsan/Ecoli10st/Analysis/Parsnp exists, all results will be overwritten 11:37:45 - INFO -

SETTINGS: |-refgenome: /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/GCA_000005845.2_ASM584v2_genomic.fna |-genomes: /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_035064325.1_ASM3506432v1_genomic.fna /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_006542905.1_ASM654290v1_genomic.fna ...728 more file(s)... /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_030204775.1_ASM3020477v1_genomic.fna /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_030203635.1_ASM3020363v1_genomic.fna |-aligner: muscle |-outdir: /home/afsan/Ecoli10st/Analysis/Parsnp |-OS: Linux |-threads: 1

11:37:45 - INFO - <<Parsnp started>> 11:37:55 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_025630715.1_ASM2563071v1_genomic.fna is 1.35x longer than reference! 11:37:58 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968495.1_ASM2296849v1_genomic.fna is 1.25x longer than reference! 11:37:58 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968635.1_ASM2296863v1_genomic.fna is 1.29x longer than reference! 11:38:06 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968425.1_ASM2296842v1_genomic.fna is 1.29x longer than reference! 11:38:19 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_964059545.1_21_582_genomic.fna is 49.76x shorter than reference genome! 11:38:26 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968855.1_ASM2296885v1_genomic.fna is 1.33x longer than reference! 11:38:29 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_964059255.1_19_683_genomic.fna is 23.28x shorter than reference genome! 11:38:37 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022130545.1_ASM2213054v1_genomic.fna is 1.26x longer than reference! 11:38:46 - DEBUG - Writing .ini file 11:38:46 - INFO - Setting the partition size to 52 11:38:46 - INFO - Running partitions... 11:38:46 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/parsnpAligner.ini 11:38:46 - INFO - 0%| | 0/15 [00:00<?, ?it/s] 11:50:28 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/log/parsnp-aligner.out 11:50:28 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/log/parsnp-aligner.err 11:50:28 - DEBUG - 11:50:28 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/parsnpAligner.ini 11:50:28 - INFO - 7%|6 | 1/15 [11:41<2:43:46, 701.87s/it] 12:02:30 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/log/parsnp-aligner.out 12:02:30 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/log/parsnp-aligner.err 12:02:30 - DEBUG - 12:02:30 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000001-out/parsnpAligner.ini 12:02:30 - INFO - 13%|#3 | 2/15 [23:43<2:34:34, 713.41s/it]

TAIL 13:54:36 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000000-out/log/parsnp-aligner.out 13:54:36 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000000-out/log/parsnp-aligner.err 13:54:36 - DEBUG - 13:54:36 - INFO - 93%|#########3| 14/15 [2:15:49<09:42, 582.12s/it] 13:54:36 - INFO - Computing intersection of all partition LCBs... 13:55:06 - INFO - Partition stats: Mean bp covered = 1807625.40 Mean LCB count = 1840.60 13:55:06 - INFO - After intersection: 0 reference bases over 0 clusters 13:55:06 - INFO - Trimming partitioned XMFAs back to intersected intervals... 13:55:06 - INFO - 0%| | 0/15 [00:00<?, ?it/s] 13:57:07 - INFO - 7%|6 | 1/15 [02:00<28:03, 120.25s/it] 13:59:04 - INFO - 20%|## | 3/15 [03:57<14:56, 74.71s/it] 14:03:49 - INFO - 27%|##6 | 4/15 [08:42<27:00, 147.27s/it] 14:08:18 - INFO - 40%|#### | 6/15 [13:12<21:10, 141.16s/it] 14:11:46 - INFO - 47%|####6 | 7/15 [16:39<21:08, 158.56s/it] 14:13:12 - INFO - 53%|#####3 | 8/15 [18:05<16:11, 138.72s/it] 14:17:59 - INFO - 60%|###### | 9/15 [22:52<18:01, 180.19s/it] 14:22:41 - INFO - 67%|######6 | 10/15 [27:34<17:26, 209.36s/it] 14:27:32 - INFO - 73%|#######3 | 11/15 [32:25<15:32, 233.16s/it] 14:32:24 - INFO - 80%|######## | 12/15 [37:17<12:30, 250.28s/it] 14:36:29 - INFO - 87%|########6 | 13/15 [41:22<08:17, 248.83s/it] 14:36:50 - INFO - 93%|#########3| 14/15 [41:44<03:01, 181.36s/it] 14:37:33 - INFO - 100%|##########| 15/15 [42:26<00:00, 140.04s/it] 14:37:33 - INFO - 100%|##########| 15/15 [42:26<00:00, 169.77s/it] 14:37:33 - INFO - Merging trimmed XMFA files... 14:37:33 - INFO - 0it [00:00, ?it/s] 14:37:33 - INFO - 0it [00:00, ?it/s] 14:37:33 - DEBUG - Writing MAF file to /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.maf 14:38:15 - DEBUG - harvesttools -q -o /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.ggr -x /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.xmfa -g /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/MG1655.gbk 14:38:17 - DEBUG - 14:38:17 - DEBUG - 14:38:17 - DEBUG - 14:38:17 - DEBUG - time harvesttools -i /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.ggr -S /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.snps.mblocks 14:38:17 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/harvest-mblocks.out 14:38:17 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/harvest-mblocks.err 14:38:17 - DEBUG - 14:38:17 - INFO - Reconstructing core genome phylogeny... 14:38:17 - WARNING - Not enough SNPs to use RaxML. Attempting to use FastTree instead... 14:38:17 - DEBUG - time FastTreeMP -nt -quote -gamma -slow -boot 100 /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.snps.mblocks 14:38:17 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/fasttree.out 14:38:17 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/fasttree.err 14:38:17 - DEBUG - 14:38:17 - INFO - Aligned 733 genomes in 3.01 hours 14:38:17 - INFO - Parsnp finished! All output available in /home/afsan/Ecoli10st/Analysis/Parsnp 14:38:17 - DEBUG - Validating output directory contents

afsansarwer avatar Jun 22 '25 09:06 afsansarwer

Hlw, could you please help us to figure this out?

We are eagerly waiting for your reply.

Thank you.

On Sun, Jun 22, 2025, 3:29 PM afsansarwer @.***> wrote:

afsansarwer left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-2994073067

Working on the same project (same genomes), I could generate a tree file, but the tree shows zero height in iTOL. I have attached the output folder in this Google Drive link - https://drive.google.com/drive/folders/1Cn8iXQ5cCiazo0gxXh9oR8Ke7LkwEw5t?usp=sharing https://drive.google.com/drive/folders/1Cn8iXQ5cCiazo0gxXh9oR8Ke7LkwEw5t?usp=sharing

Here are some snippets from the terminal after running:

HEAD (parsnp-env) @.***:~$ parsnp -r /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/GCA_000005845.2_ASM584v2_genomic.fna -g /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/MG1655.gbk -d /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/ -o /home/afsan/Ecoli10st/Analysis/Parsnp -c -v -p 1 --fo 11:37:45 - INFO - |--Parsnp 2.1.4--|

11:37:45 - WARNING - Output directory /home/afsan/Ecoli10st/Analysis/Parsnp exists, all results will be overwritten 11:37:45 - INFO -

SETTINGS: |-refgenome: /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/GCA_000005845.2_ASM584v2_genomic.fna |-genomes:

/home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_035064325.1_ASM3506432v1_genomic.fna

/home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_006542905.1_ASM654290v1_genomic.fna ...728 more file(s)...

/home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_030204775.1_ASM3020477v1_genomic.fna

/home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_030203635.1_ASM3020363v1_genomic.fna |-aligner: muscle |-outdir: /home/afsan/Ecoli10st/Analysis/Parsnp |-OS: Linux |-threads: 1

11:37:45 - INFO - <> 11:37:55 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_025630715.1_ASM2563071v1_genomic.fna is 1.35x longer than reference! 11:37:58 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968495.1_ASM2296849v1_genomic.fna is 1.25x longer than reference! 11:37:58 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968635.1_ASM2296863v1_genomic.fna is 1.29x longer than reference! 11:38:06 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968425.1_ASM2296842v1_genomic.fna is 1.29x longer than reference! 11:38:19 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_964059545.1_21_582_genomic.fna is 49.76x shorter than reference genome! 11:38:26 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022968855.1_ASM2296885v1_genomic.fna is 1.33x longer than reference! 11:38:29 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_964059255.1_19_683_genomic.fna is 23.28x shorter than reference genome! 11:38:37 - WARNING - File /home/afsan/Ecoli10st/PUBMED_genomes/All_genomes/GCA_022130545.1_ASM2213054v1_genomic.fna is 1.26x longer than reference! 11:38:46 - DEBUG - Writing .ini file 11:38:46 - INFO - Setting the partition size to 52 11:38:46 - INFO - Running partitions... 11:38:46 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/parsnpAligner.ini 11:38:46 - INFO - 0%| | 0/15 [00:00<?, ?it/s] 11:50:28 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/log/parsnp-aligner.out 11:50:28 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000003-out/log/parsnp-aligner.err 11:50:28 - DEBUG - 11:50:28 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/parsnpAligner.ini 11:50:28 - INFO - 7%|6 | 1/15 [11:41<2:43:46, 701.87s/it] 12:02:30 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/log/parsnp-aligner.out 12:02:30 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000011-out/log/parsnp-aligner.err 12:02:30 - DEBUG - 12:02:30 - DEBUG - time /home/afsan/miniconda3/envs/parsnp-env/bin/bin/parsnp_core /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000001-out/parsnpAligner.ini 12:02:30 - INFO - 13%|#3 https://github.com/marbl/parsnp/issues/3 | 2/15 [23:43<2:34:34, 713.41s/it]

TAIL 13:54:36 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000000-out/log/parsnp-aligner.out 13:54:36 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/partition/chunk-0000000000-out/log/parsnp-aligner.err 13:54:36 - DEBUG - 13:54:36 - INFO - 93%|#########3 https://github.com/marbl/parsnp/issues/3| 14/15 [2:15:49<09:42, 582.12s/it] 13:54:36 - INFO - Computing intersection of all partition LCBs... 13:55:06 - INFO - Partition stats: Mean bp covered = 1807625.40 Mean LCB count = 1840.60 13:55:06 - INFO - After intersection: 0 reference bases over 0 clusters 13:55:06 - INFO - Trimming partitioned XMFAs back to intersected intervals... 13:55:06 - INFO - 0%| | 0/15 [00:00<?, ?it/s] 13:57:07 - INFO - 7%|6 | 1/15 [02:00<28:03, 120.25s/it] 13:59:04 - INFO - 20%|## | 3/15 [03:57<14:56, 74.71s/it] 14:03:49 - INFO - 27%|##6 https://github.com/marbl/parsnp/issues/6 | 4/15 [08:42<27:00, 147.27s/it] 14:08:18 - INFO - 40%|#### | 6/15 [13:12<21:10, 141.16s/it] 14:11:46 - INFO - 47%|####6 https://github.com/marbl/parsnp/issues/6 | 7/15 [16:39<21:08, 158.56s/it] 14:13:12 - INFO - 53%|#####3 https://github.com/marbl/parsnp/issues/3 | 8/15 [18:05<16:11, 138.72s/it] 14:17:59 - INFO - 60%|###### | 9/15 [22:52<18:01, 180.19s/it] 14:22:41 - INFO - 67%|######6 https://github.com/marbl/parsnp/issues/6 | 10/15 [27:34<17:26, 209.36s/it] 14:27:32 - INFO - 73%|#######3 https://github.com/marbl/parsnp/issues/3 | 11/15 [32:25<15:32, 233.16s/it] 14:32:24 - INFO - 80%|######## | 12/15 [37:17<12:30, 250.28s/it] 14:36:29 - INFO - 87%|########6 https://github.com/marbl/parsnp/issues/6 | 13/15 [41:22<08:17, 248.83s/it] 14:36:50 - INFO - 93%|#########3 https://github.com/marbl/parsnp/issues/3| 14/15 [41:44<03:01, 181.36s/it] 14:37:33 - INFO - 100%|##########| 15/15 [42:26<00:00, 140.04s/it] 14:37:33 - INFO - 100%|##########| 15/15 [42:26<00:00, 169.77s/it] 14:37:33 - INFO - Merging trimmed XMFA files... 14:37:33 - INFO - 0it [00:00, ?it/s] 14:37:33 - INFO - 0it [00:00, ?it/s] 14:37:33 - DEBUG - Writing MAF file to /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.maf 14:38:15 - DEBUG - harvesttools -q -o /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.ggr -x /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.xmfa -g /home/afsan/Ecoli10st/PUBMED_genomes/Reference_genome/MG1655.gbk 14:38:17 - DEBUG - 14:38:17 - DEBUG - 14:38:17 - DEBUG - 14:38:17 - DEBUG - time harvesttools -i /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.ggr -S /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.snps.mblocks 14:38:17 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/harvest-mblocks.out 14:38:17 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/harvest-mblocks.err 14:38:17 - DEBUG - 14:38:17 - INFO - Reconstructing core genome phylogeny... 14:38:17 - WARNING - Not enough SNPs to use RaxML. Attempting to use FastTree instead... 14:38:17 - DEBUG - time FastTreeMP -nt -quote -gamma -slow -boot 100 /home/afsan/Ecoli10st/Analysis/Parsnp/parsnp.snps.mblocks 14:38:17 - DEBUG - STDOUT was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/fasttree.out 14:38:17 - DEBUG - STDERR was piped... please see output file: /home/afsan/Ecoli10st/Analysis/Parsnp/log/fasttree.err 14:38:17 - DEBUG - 14:38:17 - INFO - Aligned 733 genomes in 3.01 hours 14:38:17 - INFO - Parsnp finished! All output available in /home/afsan/Ecoli10st/Analysis/Parsnp 14:38:17 - DEBUG - Validating output directory contents

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-2994073067, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKHXSVKR7QBPDPWI6YD3EZZPRAVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOJUGA3TGMBWG4 . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 23 '25 17:06 naifahim

@naifahim For your issue, I see in the output the critical error:

CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering

This low identity could be caused by low-quality or mislabeled genomes in NCBI (I ran into this error when working with E coli as well).

By default, Parsnp filters out genomes which are low in identity relative to the reference. However, you have disabled that functionality with -c. I'd suggest removing that option. If too many genomes are being excluded, you can try using FastANI or MASH for recruitment and lowering the ANI threshold:

  --use-ani             Use ANI for genome filtering
  --min-ani MIN_ANI     Min ANI value required to include genome
  --min-ref-cov MIN_REF_COV
                        Minimum percent of reference segments to be covered in FastANI
  --use-mash            Use mash for genome filtering
  --max-mash-dist MAX_MASH_DIST
                        Max mash distance.

For example, instead of -c, you could try

--use-ani --min-ani 90

bkille avatar Jun 25 '25 19:06 bkille

Okay i will now try it out.

One important question:

Is 4gb RAM a problem?

Sometimes linux got shut down automatically.

On Thu, Jun 26, 2025, 1:49 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-3005953270

@naifahim https://github.com/naifahim For your issue, I see in the output the critical error:

CRITICAL - Set of filtered genomes are too divergent for parsnp, please use stricter ANI thresholds for filtering

This low identity could be caused by low-quality or mislabeled genomes in NCBI (I ran into this error when working with E coli as well).

By default, Parsnp filters out genomes which are low in identity relative to the reference. However, you have disabled that functionality with -c. I'd suggest removing that option. If too many genomes are being excluded, you can try using FastANI or MASH for recruitment and lowering the ANI threshold:

--use-ani Use ANI for genome filtering --min-ani MIN_ANI Min ANI value required to include genome --min-ref-cov MIN_REF_COV Minimum percent of reference segments to be covered in FastANI --use-mash Use mash for genome filtering --max-mash-dist MAX_MASH_DIST Max mash distance.

For example, instead of -c, you could try

--use-ani --min-ani 90

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-3005953270, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKHSMCEQOU72OOBNAZL3FL4OJAVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBVHE2TGMRXGA . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 25 '25 19:06 naifahim

@afsansarwer Similarly, your output shows that the partitions share no aligned regions in common:

13:55:06 - INFO - Partition stats: Mean bp covered = 1807625.40 Mean LCB count = 1840.60
13:55:06 - INFO - After intersection: 0 reference bases over 0 clusters

If you inspect your alignment files, they are empty.

This is also due to the sequences being too divergent. Please without -c.

bkille avatar Jun 25 '25 19:06 bkille

Is 4gb RAM a problem?

Sometimes linux got shut down automatically.

That is a very little amount of RAM, especially if you are running the partitions in parallel. Please try with only one thread. You can monitor Parsnp's memory usage via top or htop

bkille avatar Jun 25 '25 19:06 bkille

Thank you again for you reply.

I would be grateful if you clarify the isse of RAM and how many genomes can we do in a single run?

  1. I have 4gb RAM, is it enough?
  2. Can i use around 1000 genomes with good quality in parsnp?

Thank you very much for your time.

Please me know about these two issues

On Thu, Jun 26, 2025, 1:54 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-3005964322

@afsansarwer https://github.com/afsansarwer Similarly, your output shows that the partitions share no aligned regions in common:

13:55:06 - INFO - Partition stats: Mean bp covered = 1807625.40 Mean LCB count = 1840.60 13:55:06 - INFO - After intersection: 0 reference bases over 0 clusters

If you inspect your alignment files, they are empty.

This is also due to the sequences being too divergent. Please without -c.

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-3005964322, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKGCVFZRSM34LQF7ISD3FL5ANAVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBVHE3DIMZSGI . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 25 '25 19:06 naifahim

The RAM requirement varies depending on the number, quality, and divergence of genomes as well as the partition size and number of concurrent partitions (i.e. -p). For 800 M. tuberculosis, we were able to use ~3Gb RAM per partition, on average.

As far as the number of genomes, yes, we have run Parsnp on thousands of genomes. Please see this table from our manuscript for some examples. Note that the results in the table are for running 8 partitions in parallel, so you can infer that by running one partition at a time, the required memory per concurrent partition is roughly total_memory/8.

bkille avatar Jun 25 '25 20:06 bkille

Thank you for your response. I am very glad to you for your prompt response.

So can you please give me the full code? Considering partitions, RAM, divergent genome?

Than i can figure it out how to proceed in selection of genomes and managing my laptop. As i am student from developing country, i am not able use high configuration device, that's why i am struggling more than one month.

I hope you can help me to solve this problem.

On Thu, Jun 26, 2025, 2:07 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-3005996510

The RAM requirement varies depending on the number, quality, and divergence of genomes as well as the partition size and number of concurrent partitions (i.e. -p). For 800 M. tuberculosis, we were able to use ~3Gb RAM per partition, on average.

As far as the number of genomes, yes, we have run Parsnp on thousands of genomes. Please see this https://academic.oup.com/view-large/459376820 table from our manuscript for some examples. Note that the results in the table are for running 8 partitions in parallel, so you can infer that by running one partition at a time, the required memory per concurrent partition is roughly total_memory/8.

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-3005996510, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKBOKWI74UNPTRTSBDT3FL6PNAVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBVHE4TMNJRGA . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 25 '25 20:06 naifahim

Hi @naifahim,

I completely understand your constraints. Please try running

parsnp -r reference.fna -d a -a c -o result --verbose  --use-mash -p 1

Also, note that it will take substantially longer with only 1 thread. Are the genomes you are analyzing all publicly available?

bkille avatar Jun 25 '25 20:06 bkille

722 fna genomes from NCBi (GCA assembled genomes) and 10 are of our own (fasta).

Should i have to include ANI in the code?

On Thu, Jun 26, 2025, 2:33 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-3006062993

Hi @naifahim https://github.com/naifahim,

I completely understand your constraints. Please try running

parsnp -r reference.fna -d a -a c -o result --verbose --use-mash -p 1

Please note that it will take substantially longer with only 1 thread. Are the genomes you are analyzing all publicly available?

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-3006062993, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKBZPNJV66TS4ODKKBT3FMBQFAVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBWGA3DEOJZGM . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 26 '25 01:06 naifahim

The command above should be fine. It uses Mash instead of FastANI. This should work better for your computer.

If you have any more questions, please feel free to send me an email. My address is on my GitHub profile page.

bkille avatar Jun 27 '25 01:06 bkille

Thank you for your help. I have use the same code using mash but it’s showing too divergent genome issue.

Then i used Ani 90. But when partition was started, it got terminated. Even linux got shut down. I do not why it’s happening.

However, in another laptop using the same code, same RAM, it has been done. He just operate linux system in the laptop, excluded windows and powershell.

But i am using linux in windows in 4gb RAM. So is it a issue? I would be glad to if you provide me solution for this.

Finally, thanks to you for your cooperation again.

On Fri, Jun 27, 2025, 8:00 AM Bryce Kille @.***> wrote:

bkille left a comment (marbl/parsnp#180) https://github.com/marbl/parsnp/issues/180#issuecomment-3011095951

The command above should be fine. It uses Mash instead of FastANI. This should work better for your computer.

If you have any more questions, please feel free to send me an email. My address is on my GitHub profile page.

— Reply to this email directly, view it on GitHub https://github.com/marbl/parsnp/issues/180#issuecomment-3011095951, or unsubscribe https://github.com/notifications/unsubscribe-auth/BP6VIKFEOKKXB7JYU2PKCJD3FSQS7AVCNFSM6AAAAAB72MJSBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMJRGA4TKOJVGE . You are receiving this because you were mentioned.Message ID: @.***>

naifahim avatar Jun 27 '25 06:06 naifahim

Hi @naifahim,

Yes, it sounds like your genomes are rather divergent. FastANI likely requires more RAM, so that may be the issue. If you'd like, feel free to send me an email and perhaps I can run the alignment for you?

bkille avatar Jul 14 '25 20:07 bkille