Segfault in Python ASGI app in MacOS

Open phijaro opened this issue 5 months ago • 1 comments

Bug Overview

When unitd is running on MacOS with --user $USER (so the application process is owned by a user that is neither root nor nobody), using it to host a Python ASGI app that makes an HTTP request on the back end causes the application process to segfault when the request is made.

Expected Behavior

I expect the request to succeed (and I have observed that it does when I run Unit under the same circumstances on e.g. Ubuntu).

Steps to Reproduce the Bug

I've constructed a minimal Unit config and example ASGI app to illustrate the bug: macos-segfault-repro.zip

To run it:

Replace the applications/bugrepro/working_directory value in unit_config.json with the path where you extracted the unit-bug-repro directory from the zipfile.
Apply the fork safety workaround: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
Start unitd: sudo --preserve-env=OBJC_DISABLE_INITIALIZE_FORK_SAFETY unitd --user "$USER" --control-user "$USER"
Configure Unit using unit_config.json: curl -X PUT --unix-socket /opt/homebrew/var/run/unit/control.sock --data-binary @unit_config.json http://localhost/config
Make a request to the app: curl localhost

When I do this, curl localhost outputs <!DOCTYPE html><title>Error 503</title><p>Error 503. and I find a line like

2025/09/05 20:51:55 [alert] 49429#1063395 app process 49731 exited on signal 11

in the Unit log.

Environment Details

Target deployment platform: local machine
Target OS: MacOS 15.6 (I've reproduced this bug on Macs with both Intel and ARM CPUs)
Version of this project or specific commit: 1.34.2
Version of any relevant project languages: Python 3.12. (These are the versions of Unit and Python installed via Homebrew).

Additional Context

No response

Sep 05 '25 20:09 phijaro

OK, I did some more digging, and I think I've figured out what's going on here.

I wasn't able to generate a core dump, but I did find the MacOS crash report (attached), and based on that it looks like this is a subtler of the same bug that the OBJC_DISABLE_INITIALIZE_FORK_SAFETY variable is there to work around. Various other people have come across it, and a workaround is to set no_proxy='*' in the environment where the Python app is running (provided you know you'll never need to use the system-configured proxy for the requests your app backend is making). This can be done in the app directly, so a minimal update to app.py that implements the workaround might be:

import os
import urllib.request


async def application(scope, receive, send):
    if scope["type"] == "lifespan":
        message = await receive()
        if message["type"] == "lifespan.startup":
            os.environ["no_proxy"] = "example.com"
    elif scope["type"] == "http":
        with urllib.request.urlopen("https://example.com"):
            pass
        await send(
            {
                "type": "http.response.start",
                "status": 204,
            }
        )
        await send({"type": "http.response.body"})

or else in the environment member of the Unit app config, so with the workaround added unit_config.json looks like this:

{
    "listeners": {
        "127.0.0.1:80": {"pass": "applications/bugrepro"}
    },
    "applications": {
        "bugrepro": {
            "type": "python",
            "module": "app",
            "path": [""],
            "working_directory": "/Users/phil.roberts/src/unit-bug-repro",
            "environment": {"no_proxy": "example.com"}
        }
    }
}

Some references:

https://wefearchange.org/2018/11/forkmacos.rst
https://github.com/python/cpython/issues/74570#issuecomment-1093748533

This is effectively a bug in Python rather than Unit, but the way Unit runs Python apps (using a forked process) is likely to trigger it. (I should note that it's not just ASGI apps: I was able to reproduce the same bug in a minimal WSGI app too).

unit-crash.txt

Sep 16 '25 20:09 phijaro