ERROR: Cannot run diamond

[shiyi-pan]

Hi, David：

I want to use OrthoFinder with diamond v0.9.32.133 to do some research, but have a problem.here is my script:

export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/OrthoFinder_source/scripts_of/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/mafft/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/FastME/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/mcl/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/python36/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/ruanjian/iqtree-1.6.12-Linux/bin/:$PATH export PATH=/gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/:$PATH

python $ORTHOFINDER/orthofinder.py -t 24 -a 24 -S diamond -M msa -A mafft -I 1.5 -T fasttree -f /gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/inputfasta

and here is the error I meet:

OrthoFinder version 2.4.1 Copyright (C) 2014 David Emms

2021-10-10 17:29:22 : Starting OrthoFinder 2.4.1 24 thread(s) for highly parallel tasks (BLAST searches etc.) 24 thread(s) for OrthoFinder algorithm

Checking required programs are installed

ERROR: Cannot run diamond Format of make database command: /gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/diamond makedb --in INPUT -d OUTPUT ERROR: Cannot run diamond Format of search database command: /gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/diamond blastp -ignore-warnings -d DATABASE -q INPUT -o OUTPUT --more-sensitive -p 1 --quiet -e 0.001 --compress 1 Please check diamond is installed and that the executables are in the system path

ERROR: An error occurred, please review the error messages they may contain useful information about the problem.

I can run diamond alone like this:

[hjb20181119@login 02.SoybeanData]$ diamond makedb --in SoyC01.longest.Protein.fa -d . diamond v0.9.32.133 (C) Max Planck Society for the Advancement of Science Documentation, support and updates available at http://www.diamondsearch.org

#CPU threads: 40 Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1) Database input file: SoyC01.longest.Protein.fa Opening the database file... [0.062s] Loading sequences... [0.109s] Masking sequences... [0.111s] Writing sequences... [0.091s] Hashing sequences... [0.006s] Loading sequences... [0s] Writing trailer... [0s] Closing the input file... [0s] Closing the database file... [0.001s] Database hash = 907065fcd3662d0c2016b62f071ae28f Processed 54405 sequences, 20025936 letters. Total time = 0.384s

and I write it into environment variable：

export PATH=/gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/:$PATH

I also write the absolute path into config.json file:

"diamond":{
"program_type": "search",
"db_cmd": "/gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/diamond makedb --in INPUT -d OUTPUT",
"search_cmd": "/gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/diamond blastp -ignore-warnings -d DATABASE -q INPUT -o OUTPUT --more-sensitive -p 1 --quiet -e 0.001 --compress 1"
},

I can run orthofinder with NCBI-blast,too . This is confuse me a lot , could you help me fix it? thank you very.

[sjfleck]

I'm having a similar problem. When I submit my script to our cluster, I get the same errors as you (i.e. "ERROR: Cannot run diamond"), but when I run the same exact commands in my front end (in terminal, where I can't actually run anything because of a memory, cpu, thread, and time limits), it seems to work perfectly fine ...except where it'll be terminated prematurely due to the restrictions mentioned above.

If you're running jobs on a cluster, maybe we have the same issue. If I figure out my problem, I'll share it here. It could be as simple as a problem with permissions.

[sjfleck]

I have an update. I decided to run this on a different partition in our cluster and it seems to be working without any issues:

OrthoFinder version 2.5.4 Copyright (C) 2014 David Emms

2021-10-21 12:05:12 : Starting OrthoFinder 2.5.4 16 thread(s) for highly parallel tasks (BLAST searches etc.) 16 thread(s) for OrthoFinder algorithm

Checking required programs are installed Test can run "mcl -h" - ok

Dividing up work for BLAST for parallel processing 2021-10-21 12:05:39 : Creating diamond database 1 of 9 2021-10-21 12:05:40 : Creating diamond database 2 of 9 2021-10-21 12:05:40 : Creating diamond database 3 of 9 2021-10-21 12:05:40 : Creating diamond database 4 of 9 2021-10-21 12:05:41 : Creating diamond database 5 of 9 2021-10-21 12:05:41 : Creating diamond database 6 of 9 2021-10-21 12:05:41 : Creating diamond database 7 of 9 2021-10-21 12:05:42 : Creating diamond database 8 of 9 2021-10-21 12:05:42 : Creating diamond database 9 of 9

Running diamond all-versus-all Using 16 thread(s) 2021-10-21 12:05:43 : This may take some time.... 2021-10-21 12:05:43 : Done 0 of 81 2021-10-21 12:18:33 : Done 10 of 81

The partition that didn't work was our "debug" partition. This is a partition with very few nodes and a 1 hr runtime limit where we can submit jobs in order to make sure they're working properly before we start requesting more resources in our "general-compute" partition. For some reason, the same script fails in "debug" but runs fine in "general-compute". It seems like this is an error on my end and I've contacted our cluster's help desk about it. Sorry for taking up your time and I hope you figured out your issue!

[davidemms]

Hi @shiyi-pan

In the message it also says that it is tryign to run this search command, does it work if you run it outside OrthoFinder?

/gss1/home/hjb20181119/panyongpeng/NN1138-2/03.orthofinder_data/02.SoybeanData/diamond/diamond blastp -ignore-warnings -d DATABASE -q INPUT -o OUTPUT --more-sensitive -p 1 --quiet -e 0.001 --compress 1

Best wishes David

[shiyi-pan]

Hi, David I'm sorry I reply you so lately. My diamond works well outside OrthoFinder. After that, I just remove the diamond in script_of/bin path and install a now diamond and put it in PATH variable. Somehow , orthofinder works. I don't know how it works , maybe other researchers and you could understand it.

Best wishes shiyi

[cymon]

Hi shiyi-pan,

Check that the version of diamond that is in Orthofinder/bin works on your OS.

Despite what the error message implied, Orthofinder does not search your system $PATH for a working copy of diamond. If, as was my case, the Orthofinder supplied version of diamond in Orthofinder/bin is broken, just make a symbolic link to a working version of diamond in that directory.

Cheers, C.