Various data races in subinterpreter tests under TSAN

[colesbury]

Bug report

Thread sanitizer reports a number of data races when running subinterpreter tests

InterpreterPoolMixin (test_concurrent_futures)

• [ ] run_eval_code_obj: see https://github.com/python/cpython/issues/128130
• [ ] posixmodule_exec modifies globals waitid_result_desc and stat_result_desc
• [x] _PyBuiltins_AddExceptions: modifies globals PyExc_EnvironmentError, etc. https://github.com/python/cpython/blob/49bd47d5f14993d37b97aa2bbf257f5df16b96a9/Objects/exceptions.c#L4354-L4355
• [ ] _PyInterpreterState_FinalizeAllocatedBlocks: modifies _PyRuntime variable _PyInterpreterState_FinalizeAllocatedBlocks
• [ ] _globals_fini + clear_interpreter: data race on _globals.module_count (the read in clear_interpreter happens outside of a lock)
• [ ] init_static_exctypes: modifies globals _PyExc_InterpreterError, etc.
• [ ] _structmodule_exec: data race reported on ptr->unpack = native->unpack

I think for now it would make sense to skip the InterpreterPoolMixin tests when running with TSAN.

test__interpchannels

• [ ] find_name_in_mro and _waiting_release: I don't understand this one, but it happens during test_send_recv_different_interpreters_and_threads

test__interpreters and test_interpreters

• [ ] type_ready_set_new: https://github.com/python/cpython/blob/7b2e01bb555359913939d7ff168363f1760d3f8e/Objects/typeobject.c#L8509
• [ ] type_ready: (see https://github.com/python/cpython/issues/129817)
• [ ] File descriptor race in test_api.LowLevelTests.test_is_running and test_running: close() concurrent with read() on the same file descriptor.
• [ ] managed_static_type_state_init in test.test_interpreters.test_stress.StressTests.test_create_many_threaded: data race on the non-atomic read of interp_count.
• [x] pegen memo_statistics mutex is currently only used in free threading build (seen in test_create_many_threaded)
• [x] version tags on static types: NEXT_GLOBAL_VERSION_TAG, type->tp_versions_used, and tp->tp_version_tag modifications are not thread-safe (seen in test_create_many_threaded)

test_capi.test_misc.assign_version_tag

• [x] race on NEXT_GLOBAL_VERSION_TAG in assign_version_tag (see above)

Linked PRs

• gh-129826
• gh-129829
• gh-135794
• gh-136266
• gh-143446

[ZeroIntensity]

Are you still working on this? I'm happy to take a stab at some of these.

[colesbury]

No, I'm not actively working on them. Go for it.