uwsgi icon indicating copy to clipboard operation
uwsgi copied to clipboard

Added log before waiting on threads

Open jafar-atili opened this issue 1 year ago • 1 comments

Why do we need this log?

We manage a large Python application. once we started using the max-requests we noticed that clients are receiving 502 during the max-request reload.

It is worth mentioning that we have 5x uwsgi processes, each with 70x threads.

Our first step toward resolving this behavior was adding the reload-mercy/worker-reload-mercy to grant our application 20 minutes to finish up the old requests and gracefully shut down the worker.

This still didn't resolve our problem, and here we wanted to understand what is preventing our workers from shutting down gracefully.

In order to investigate it, we wanted to understand what happens inside our application before the reload to understand what is stuck and prevent our application from shutting down gracefully.

we compiled a version of uwsgi with this PR, and once we saw the new log message we knew this is our window (until the mercy-reload timeout) to dump the process stacktrace (we used py-spy) from the host.

jafar-atili avatar Aug 22 '23 07:08 jafar-atili

max-requests-delta will help you. Although this is 3.0 only option.

methane avatar Mar 19 '24 15:03 methane