meshed-memory-transformer icon indicating copy to clipboard operation
meshed-memory-transformer copied to clipboard

OSError: [Errno 24] Too many open files

Open 775266553 opened this issue 8 months ago • 0 comments

File "train.py", line 100, in train_scst tokenizer_pool = multiprocessing.Pool() File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/context.py", line 119, in Pool context=self.get_context()) File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/pool.py", line 174, in init self._repopulate_pool() File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/pool.py", line 239, in _repopulate_pool w.start() File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/root/anaconda3/envs/m2/lib/python3.6/multiprocessing/popen_fork.py", line 65, in _launch parent_r, child_w = os.pipe() OSError: [Errno 24] Too many open files

问题:运行到强化学习阶段,系统报错 Too many open files 。

Problem: Running into the intensive learning phase, the system reports an error.

775266553 avatar Oct 20 '23 00:10 775266553