Problems with limits on the number of files even after updating them

[JuG-git]

Hi!

I’m trying to run an Orthofinder analysis on 1006 Saccharomyces cerevisiae genomes and we are having problems with soft and hard limits. It is quite likely that the problem is on our server since linux is not reloading the logged area. Still we would like to know if anybody has seeing something like this before.

When I started the run I had this problem and just as the error suggests, it was possible to reset the soft limit using ulimit -n and opening a new session. While running species tree Ortho identified the number of files problem once again. I tried to workaround with linux command su, as David suggested latter on issue #384 but it made no difference. Our root also edited our limits on limits.conf (the ulimit commands confirm that the limits have been changed) and still we could not run the program. Has anybody experienced something like this before?

Thanks in advance.

[damizuka]

Hi, I have the same problem currently I'm working with 566 files, the ulimit was set to 1000000 for soft and hard limits. Still de same error: OrthoFinder version 2.5.4 Copyright (C) 2014 David Emms

2022-04-11 17:22:49 : Starting OrthoFinder 2.5.4 25 thread(s) for highly parallel tasks (BLAST searches etc.) 25 thread(s) for OrthoFinder algorithm

Checking required programs are installed

Test can run "mcl -h" - ok Test can run "mafft /home/dgonzalez/nueva_carpeta/OrthoFinder/Results_Apr11/WorkingDirectory//_dependencies_check/SimpleTest.fa" - ok Test can run "FastTree /home/dgonzalez/nueva_carpeta/OrthoFinder/Results_Apr11/WorkingDirectory//_dependencies_check/SimpleTest.fa" - ok ERROR: Attempted to open required files for OrthoFinder run but an unexpected error occurred.

Stacktrace: Traceback (most recent call last): File "/home/dgonzalez/anaconda3/envs/orthofinder/bin/orthofinder", line 7, in main(args) File "/home/dgonzalez/anaconda3/envs/orthofinder/bin/scripts_of/main.py", line 1763, in main options = CheckOptions(options, speciesInfoObj.speciesToUse) File "/home/dgonzalez/anaconda3/envs/orthofinder/bin/scripts_of/main.py", line 1695, in CheckOptions fh.append(open(fnij, 'w')) OSError: [Errno 23] Too many open files in system: '/home/dgonzalez/nueva_carpeta/OrthoFinder/Results_Apr11/WorkingDirectory/Files_test/Sp170/Sp396.txt'

[davidemms]

Hi

The code is just trying to open that number of files as a test and it's getting an operating system error, so it looks like a limit on your system. The problem would also be very easy to reproduce with just a for loop in python attempting to open the required number of files, which may help you diagnose what's happening.

Best wishes David