OSError: [Errno 24] Too many open files

[nbargues]

Hi, apparently I have a multiprocess error when running adVNTR ; the command :

advntr genotype --alignment_file $bam --working_directory $advntrDir/$bam_name --vntr_id 25561 --pacbio --frameshift -m ../hg19_selected_VNTRs_Pacbio.db -t $Ncpu

the error :

[M::bam2fq_mainloop] discarded 0 singletons [M::bam2fq_mainloop] processed 14991 reads Process Process-1009: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod self._connect() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client c = SocketClient(address) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Process Process-1017: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod self._connect() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client c = SocketClient(address) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Process Process-1010: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod self._connect() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client c = SocketClient(address) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Process Process-994: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod self._connect() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client c = SocketClient(address) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Process Process-1003: Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 749, in _callmethod conn = self._tls.connection AttributeError: 'ForkAwareLocal' object has no attribute 'connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 264, in check_if_pacbio_read_spans_vntr self.check_if_flanking_regions_align_to_str(str(read.seq).upper(), length_distribution, spanning_reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 260, in check_if_flanking_regions_align_to_str spanning_reads.append(read_str[left_align[3]:right_align[3]+flanking_region_size]) File "", line 2, in append File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 753, in _callmethod self._connect() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/managers.py", line 740, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 487, in Client c = SocketClient(address) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/connection.py", line 614, in SocketClient s.connect(address) FileNotFoundError: [Errno 2] No such file or directory Traceback (most recent call last): File "/opt/miniconda3/envs/gene36/bin/advntr", line 11, in sys.exit(main()) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/main.py", line 121, in main genotype(args, genotype_parser) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/advntr_commands.py", line 101, in genotype genome_analyzier.find_repeat_counts_from_pacbio_alignment_file(input_file) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/genome_analyzer.py", line 103, in find_repeat_counts_from_pacbio_alignment_file copy_numbers = self.vntr_finder[vid].find_repeat_count_from_pacbio_alignment_file(alignment_file, reads) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/profiler.py", line 8, in wrapper retval = func(*args, **kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 477, in find_repeat_count_from_pacbio_alignment_file mapped_spanning_reads = self.get_spanning_reads_of_aligned_pacbio_reads(alignment_file) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/profiler.py", line 8, in wrapper retval = func(*args, **kwargs) File "/opt/miniconda3/envs/gene36/lib/python3.6/site-packages/advntr/vntr_finder.py", line 329, in get_spanning_reads_of_aligned_pacbio_reads p.start() File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/opt/miniconda3/envs/gene36/lib/python3.6/multiprocessing/popen_fork.py", line 65, in _launch parent_r, child_w = os.pipe() OSError: [Errno 24] Too many open files