cpython
cpython copied to clipboard
Published
20 hours ago •
python
python
Make TSAN tests pass with the GIL disabled in free-threaded builds
Feature or enhancement
We need to make the TSAN tests pass with the GIL disabled before we can disable the GIL by default in free-threaded builds.
This should proceed in two phases:
- Add suppressions for existing TSAN warnings.
- Triage, fix, and remove the suppressions for the warnings enumerated in (1).
How to run the TSAN tests
- Build with TSAN:
env CC=clang CXX=clang++ ./configure --disable-gil --with-thread-sanitizer --with-pydebug && make -j - Run tests:
env TSAN_OPTIONS=suppressions=<repo_root>/Tools/tsan/suppressions_free_threading.txt ./python -m test --tsan -j4
Working on a race
- Verify that the TSAN tests are passing using the steps from above.
- Pick a suppression from the section below and assign it to yourself (edit it and add your username next to it). Some of them may be related. If you find that other suppressions are related to the race you're working on please assign them to yourself or contact their owner if they're already assigned.
- Optionally, head over to the race index to see examples and which tests reproduce the race corresponding to the suppression.
- Delete the suppression from
<repo_root>/Tools/tsan/suppressions_free_threading.txt, run the TSAN tests, and verify that the race is reported by TSAN. You may need to comment out unrelated functions (notably,_PyEval_EvalFrameDefault) in order to reproduce the race. - Fix the race.
Suppressions
### Tasks
- [ ] _add_to_weak_set
- [ ] _in_weak_set
- [ ] _PyEval_EvalFrameDefault
- [ ] _PyType_HasFeature
- [ ] assign_version_tag
- [ ] insertdict - @SonicField
- [ ] lookup_tp_dict - @SonicField
- [ ] PyMember_GetOne - @dpdani
- [ ] PyMember_SetOne - @dpdani
- [ ] new_reference
- [x] set_contains_key - @aisk
- [ ] set_discard_entry
- [ ] set_inheritable - @colesbury
- [ ] Py_SET_TYPE
- [ ] _PyDict_CheckConsistency
- [ ] _Py_dict_lookup_threadsafe
- [ ] _multiprocessing_SemLock_acquire_impl
- [ ] dictiter_new
- [ ] dictresize
- [ ] insert_to_emptydict
- [ ] list_get_item_ref
- [ ] make_pending_calls
- [ ] set_add_entry
- [ ] _Py_slot_tp_getattr_hook
- [ ] add_threadstate
- [ ] dump_traceback
- [ ] fatal_error
- [ ] _multiprocessing_SemLock_release_impl
- [ ] _PyFrame_GetCode
- [ ] _PyFrame_Initialize
- [ ] PyInterpreterState_ThreadHead
- [ ] _PyObject_TryGetInstanceAttribute
- [ ] PyThreadState_Next
- [ ] Py_TYPE
- [ ] PyUnstable_InterpreterFrame_GetLine
- [x] sock_close - @aisk
- [ ] tstate_delete_common
- [ ] tstate_is_freed
- [ ] free_threadstate
- [ ] type_modified_unlocked
- [ ] write_thread_id
- [ ] PyThreadState_Clear
- [x] _PyImport_AcquireLock - @colesbury
- [x] _PyImport_ReleaseLock - @colesbury
- [x] _imp_release_lock - @colesbury
- [x] builtin_compile_impl - @colesbury
- [x] should_intern_string - @colesbury
- [x] _mi_heap_delayed_free_partial - @colesbury
- [x] mi_heap_visit_pages - @colesbury
- [x] mi_page_decode_padding - @colesbury
- [x] gc_restore_tid - @colesbury
- [x] _PyObject_GC_IS_SHARED - @mpage
- [x] _PyObject_GC_SET_SHARED - @mpage
- [x] _PyObject_GC_TRACK - @mpage
- [x] initialize_new_array - @SonicField
- [x] _PyFunction_SetVersion - @mpage
- [x] compare_unicode_unicode
- [x] unicode_hash -@corona10
- [x] dictkeys_decref - @DinoV
- [x] dictkeys_incref - @DinoV
Linked PRs
- gh-117736
- gh-117828
- gh-117830
- gh-117954
- gh-117955
- gh-118064
- gh-118067
- gh-118111
- gh-118165
- gh-118200
- gh-118249
- gh-118256
- gh-118258
- gh-118260
- gh-118261
- gh-118292
- gh-118301
- gh-118313
- gh-118349
- gh-118523
- gh-118533
- gh-118722
- gh-118747
- gh-118769
- gh-118796
- gh-118812
- gh-118842
- gh-118843
- gh-118856
- gh-118865
- gh-118870
- gh-118904
- gh-118905
- gh-118931
- gh-119267
- gh-119268
- gh-119312
- gh-119368
- gh-119416
- gh-119417
- gh-119883
- gh-119887
- gh-119903
- gh-119904
- gh-119907
- gh-119908
- gh-119914
- gh-119915
- gh-119921
- gh-119923
- gh-119927
- gh-119939
- gh-119992
- gh-120005
- gh-120007
- gh-120038
- gh-120118
- gh-120165
- gh-120169
- gh-120175
- gh-120210
- gh-120403
- gh-120443
- gh-120444
- gh-120458
- gh-120484
- gh-120490
- gh-120500
- gh-120510
- gh-120542
- gh-120554
- gh-120559
- gh-120654
- gh-120655
- gh-120502
- gh-120914
- gh-121240
- gh-121508
- gh-121511
- gh-121541
- gh-121549
- gh-121551
- gh-121553
- gh-121590
- gh-121598
- gh-121599
- gh-121748
I'll pick up insertdict and lookup_tp_dict now I have initialize_new_array done. I'll bundle all three into a pull request. Sounds reasonable?
@SonicField - great! Would you please mark the initialize_new_array as "ready"?
