`capfd` appears to be flaky on Windows (sometimes returning an empty string)

[wjakob]

The nanobind test suite occasionally runs into a flake (*) where capfd does not correctly capture output printed to stderr. The structure of the test looks as follows:

def test_01(capfd):
    c_function()
    captured = capfd.readouterr()
    assert captured.err.strip() == "<some string>"

Here, c_function() is a function implemented outside of Python (bound via the C API) that invokes

fprintf(stderr, "<some string>\n");

This works reliably on Linux and on macOS. On Windows, every once in a while, the test fails, and the contents of captured.err.strip() are empty.

In the linked error report, testing took place on GitHub actions with Python 3.10 and PyTest 7.2.2. When the flake occurs, it is usually enough to re-run the test and it passes.

I am quite sure that the capfd or I/O system are somehow to blame -- we previously used Python warnings to capture the output, which was rock-solid for many months. It was only once I started switching to the capfd/fprintf combination that the flaky behavior started to emerge.

I would be grateful if you have ideas.

Many thanks! Wenzel

(*) for example, this one: https://github.com/wjakob/nanobind/actions/runs/4533051238