no progress on a very large collection of proteins

Open rocpengliu opened this issue 2 years ago • 0 comments

Hi, David

First, thank you for developing orthofinder.

my case is a very weird. I have a collection of ~28 million protein sequences from ~8000 bacterial species. i run orthofinder on April 7, and so far there is no much progress. here is my script: OrthoFinder/orthofinder -f seqs/ -a 15 on a server with ~700 GB's memory and 70 threads

here are the last several lines of Log.txt in Results_Apr07

7948: zmp_info.fasta 7949: zmr_info.fasta 7950: zpa_info.fasta 7951: zpl_info.fasta 7952: zpr_info.fasta 7953: zro_info.fasta 7954: zsp_info.fasta 7955: ztr_info.fasta

here are the infor in WorkingDirectory

-rw-rw-r-- 1 peng peng 168457 Apr 7 13:49 SpeciesIDs.txt -rw-rw-r-- 1 peng peng 4924623 Apr 7 13:49 Species7955.fa -rw-rw-r-- 1 peng peng 640589749 Apr 7 13:49 SequenceIDs.txt drwxrwxr-x 3 peng peng 274432 Apr 7 13:49 ./ drwxrwxr-x 7958 peng peng 176128 Apr 7 14:31 Files_test/

here are the infor in Files_test

-rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra17.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra16.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra15.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra14.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra13.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra12.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra11.txt -rw-rw-r-- 1 peng peng 0 Apr 7 14:31 Extra10.txt

it seems orthofinder is still running, but the progress just paused on April 7, (i used htop to check it)

i am wondering if you have any ideas about this?

By the way, i have a test of a small dataset, about 400 species, 1038337 protein sequences, and there is no probelm. i also had a run on ~ 700 species with ~5million protein sequence 1 year ago. there was also no problem although it took about 2 weeks to finish.

thank you so much!

Peng

Apr 13 '23 15:04 rocpengliu