loguru icon indicating copy to clipboard operation
loguru copied to clipboard

lastest version 0.7.0 still deadlock,there was no runtime error raised

Open ShuHu2015 opened this issue 1 year ago • 8 comments

image Loguru is deadlocked until it receives a signal from celery before exiting, I use 0.7.0 version. For me, I accept there is a runtime error raised, but it is a deadlock

ShuHu2015 avatar Aug 28 '23 09:08 ShuHu2015

Hi.

Do you have access to the full traceback (including the Exception type)? What makes you think you're facing a deadlock?

Delgan avatar Aug 28 '23 09:08 Delgan

Hi.

Do you have access to the full traceback (including the Exception type)? What makes you think you're facing a deadlock?

Loguru is deadlocked until it receives a signal from celery before exiting, my celery task max time out is 2 hours. and the trackback means it is stuck in self._queue.put(str_record), sorry for my bad english. image

ShuHu2015 avatar Aug 28 '23 09:08 ShuHu2015

and sorry i cant make a simple demo to reproduce it

ShuHu2015 avatar Aug 28 '23 09:08 ShuHu2015

Thanks. I'm working on a new implementation of the enqueue parameter that should hopefully avoid this kind of issue. I'll let you know when it's published.

Delgan avatar Aug 28 '23 09:08 Delgan

Thanks. I'm working on a new implementation of the enqueue parameter that should hopefully avoid this kind of issue. I'll let you know when it's published.

it will be nice if there is a workaround to avoid this issue now, thanks.

ShuHu2015 avatar Aug 28 '23 09:08 ShuHu2015

Sorry, I can't really think of any workaround except not using enqueue=True. :confused:

Delgan avatar Aug 28 '23 10:08 Delgan

I want to understand this deadlock better, it seems deadlocked in simplequeue class self._wlock, What kind of situation can cause wlock deadlock, thanks a lot for your help!

def put(self, obj):

      # serialize the data before acquiring the lock
      obj = _ForkingPickler.dumps(obj)
      if self._wlock is None:
          # writes to a message oriented win32 pipe are atomic
          self._writer.send_bytes(obj)
      else:
          with self._wlock:
              self._writer.send_bytes(obj)](url)

ShuHu2015 avatar Aug 29 '23 01:08 ShuHu2015

Actually I'm not aware of any true deadlock. However, I know that self._writer.send_bytes() can block for a long time if the reader (on the the other side of the SimpleQueue) does not process the messages fast enough. This happens because the queue has a limited size of 65 kbytes. If you configured a slow sink, the queue can fill up and eventually blocks until a free slot is available.

Delgan avatar Aug 29 '23 07:08 Delgan