OSError: [Errno 24] Too many open files

[RonaldinhoL]

now i got a new error,

  File "/home/ronaldinho/xxx/xxx/venv/lib/python3.9/site-packages/loguru/_logger.py", line 939, in add
    handler = Handler(
  File "/home/ronaldinho/xxx/xxx/venv/lib/python3.9/site-packages/loguru/_handler.py", line 86, in __init__
    self._queue = multiprocessing.SimpleQueue()
  File "/home/ronaldinho/.pyenv/versions/3.9.4/lib/python3.9/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/home/ronaldinho/.pyenv/versions/3.9.4/lib/python3.9/multiprocessing/queues.py", line 342, in __init__
    self._rlock = ctx.Lock()
  File "/home/ronaldinho/.pyenv/versions/3.9.4/lib/python3.9/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/home/ronaldinho/.pyenv/versions/3.9.4/lib/python3.9/multiprocessing/synchronize.py", line 162, in __init__
  File "/home/ronaldinho/.pyenv/versions/3.9.4/lib/python3.9/multiprocessing/synchronize.py", line 57, in __init__
OSError: [Errno 24] Too many open files

i am use 100 task only, each task have a logger instance

ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 205441
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 205441
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

so what dose it triggered? is it open files over 1024?

[Delgan]

Do you have any code snippet to share so I can try to reproduce it on my own computer please?

[RonaldinhoL]

i just use:

self._logger = copy.deepcopy(base_logger)
self._logger.add(EXEC_DIR.joinpath(f"./logs/{logfilename}.log"),
                 encoding="utf8", retention="3 days", rotation="32 MB", compression="zip", enqueue=True)

also, my program is a network program, dose tcp port occupy file handle too?

[RonaldinhoL]

when will logger instance release file handle?

[Delgan]

when will logger instance release file handle?

You can remove() the added handler, it should release the file handle. Let me know if that solve your issue.