uwsgi
uwsgi copied to clipboard
Added log before waiting on threads
Why do we need this log?
We manage a large Python application. once we started using the max-requests
we noticed that clients are receiving 502
during the max-request
reload.
It is worth mentioning that we have 5x uwsgi
processes, each with 70x threads.
Our first step toward resolving this behavior was adding the reload-mercy
/worker-reload-mercy
to grant our application 20 minutes to finish up the old requests and gracefully shut down the worker.
This still didn't resolve our problem, and here we wanted to understand what is preventing our workers from shutting down gracefully.
In order to investigate it, we wanted to understand what happens inside our application before the reload to understand what is stuck and prevent our application from shutting down gracefully.
we compiled a version of uwsgi
with this PR, and once we saw the new log message we knew this is our window (until the mercy-reload
timeout) to dump the process stacktrace
(we used py-spy
) from the host.
max-requests-delta will help you. Although this is 3.0 only option.