Simplified worker monitoring and restart, without gunicorn.

[gnat]

Drastically simplifies app deployment for Starlette and FastAPI for many users.

Survive worker crashes directly in Uvicorn- go ahead and run your ffmpeg and machine learning tasks. Uvicorn will auto restart your workers up to the desired --workers count, reliably, even under heavy load.

• For many users, this removes gunicorn as a dependency. @tomchristie @tiangolo
  • https://github.com/encode/uvicorn/issues/517#issuecomment-564090865
  • Simplifies production documentation for many scenarios.
• Tested, benchmarked on both Windows 11 :window: and Linux! @euri10 @kludex
• Under 20 lines, backwards compatible, no effect on uvicorn performance.
• You can now "reload" your app by killing all of the child processes. :stuck_out_tongue:

Related issues

• Closes: https://github.com/encode/uvicorn/issues/517
• Good resolution for: https://github.com/encode/starlette/issues/671
• Many tangentially related FastAPI issues which would have never been reported if this was included in Uvicorn.

Deployments right now...

caddy/nginx ➡️ gunicorn ➡️ uvicorn ➡️ starlette/fastapi

For many users can be simplified to...

caddy/nginx ➡️ uvicorn ➡️ starlette/fastapi

Stress tested to 100,000's of r/s using using hey.

Thoughts, feedback appreciated.

I realise this isn't as "feature rich" as some may want (handling various signals, etc), but we're eliminating an entire dependency with only a few lines. It works well and is a relatively tiny change for big benefits, and can be easily re-factored out if something better is implemented in the future.

[humrochagf]

There's a behavior change that may be unexpected for those running on k8s and expect the pod to be restarted during a crash. To cover those, I recommend having an unmanaged option

[gnat]

I would agree, but I'm not sure it matters because this new functionality is only present if --workers=2 or greater. (multiprocess.py only)

Unless I'm mistaken, k8s pods are running single worker mode (1 worker = 1 pod), because k8s is being used as the "worker manager".

In the current multiprocess.py behavior: the parent uvicorn process awkwardly stays alive (and unusable) even when all workers are dead: #517 This mode has never been suitable for use with an external worker manager, AFAIK.

[humrochagf]

I would agree, but I'm not sure it matters because this new functionality is only present if --workers=2 or greater. (multiprocess.py only)

Unless I'm mistaken, k8s pods are running single worker mode (1 worker = 1 pod), because k8s is being used as the "worker manager".

In the current multiprocess.py behavior: the parent uvicorn process awkwardly stays alive (and unusable) even when all workers are dead: #517 This mode has never been suitable for use with an external worker manager, AFAIK.

Sorry my bad, you are right, the multiprocess isn't used in the single worker case

[zanieb]

2. Answer the question: why this shouldn't be an opt-in feature?

It seems like advanced monitoring should be opt-in and simple "is alive" monitoring should be the default since Python doesn't let you "opt-out" of dependencies.

[gnat]

2. Answer the question: why this shouldn't be an opt-in feature?

The current behavior is broken: multiprocess.py becomes a zombie process when all the workers are dead.

multiprocess.py is not used when running under k8s or gunicorn, so it's not obvious that this is broken.

[Kludex]

We're not doing this. I prefer to have reliable restart, as I said on https://github.com/encode/uvicorn/pull/1942#discussion_r1167559989.

I'll make @abersheeran 's PR happen instead: https://github.com/encode/uvicorn/pull/2183 👍