Various data races in subinterpreter tests under TSAN

[colesbury]

Bug report

Thread sanitizer reports a number of data races when running subinterpreter tests

InterpreterPoolMixin (test_concurrent_futures)

• run_eval_code_obj: see Race on _PyRuntime.signals.unhandled_keyboard_interrupt when calling eval() concurrently in free-threading mode #128130
• posixmodule_exec modifies globals waitid_result_desc and stat_result_desc
• _PyBuiltins_AddExceptions: modifies globals PyExc_EnvironmentError, etc.

  cpython/Objects/exceptions.c

  Lines 4354 to 4355 in 49bd47d

  	INIT_ALIAS(EnvironmentError, OSError);
  	INIT_ALIAS(IOError, OSError);

• _PyInterpreterState_FinalizeAllocatedBlocks: modifies _PyRuntime variable _PyInterpreterState_FinalizeAllocatedBlocks
• _globals_fini + clear_interpreter: data race on _globals.module_count (the read in clear_interpreter happens outside of a lock)
• init_static_exctypes: modifies globals _PyExc_InterpreterError, etc.
• _structmodule_exec: data race reported on ptr->unpack = native->unpack

I think for now it would make sense to skip the InterpreterPoolMixin tests when running with TSAN.

test__interpchannels

• find_name_in_mro and _waiting_release: I don't understand this one, but it happens during test_send_recv_different_interpreters_and_threads

test__interpreters and test_interpreters

• type_ready_set_new:

  cpython/Objects/typeobject.c

  Line 8509 in 7b2e01b

  type->tp_flags |= Py_TPFLAGS_DISALLOW_INSTANTIATION;

• type_ready: (see Data race on tp_flags with subinterpreters and static types #129817)
• File descriptor race in test_api.LowLevelTests.test_is_running and test_running: close() concurrent with read() on the same file descriptor.
• managed_static_type_state_init in test.test_interpreters.test_stress.StressTests.test_create_many_threaded: data race on the non-atomic read of interp_count.
• pegen memo_statistics mutex is currently only used in free threading build (seen in test_create_many_threaded)
• version tags on static types: NEXT_GLOBAL_VERSION_TAG, type->tp_versions_used, and tp->tp_version_tag modifications are not thread-safe (seen in test_create_many_threaded)

test_capi.test_misc.assign_version_tag

• race on NEXT_GLOBAL_VERSION_TAG in assign_version_tag (see above)

Linked PRs

• gh-129824: Temporarily skip InterpreterPoolMixin tests under TSAN #129826
• gh-129824: Fix some data races with types in subinterpreters #129829

[ZeroIntensity]

Are you still working on this? I'm happy to take a stab at some of these.

[colesbury]

No, I'm not actively working on them. Go for it.