TSAN complains on free-threaded list operations

[tom-pytel]

Bug report

Bug description:

Discovered a case where TSAN complains about list operations in free-threaded build and posting it here on the suggestion of @colesbury.

Reproducer:

import threading


def copy_back_and_forth(b, a, count):
    b.wait()
    for _ in range(count):
        a[0] = a[1]
        a[1] = a[0]


def check(funcs, *args):
    barrier = threading.Barrier(len(funcs))
    thrds = []

    for func in funcs:
        thrd = threading.Thread(target=func, args=(barrier, *args))

        thrds.append(thrd)
        thrd.start()

    for thrd in thrds:
        thrd.join()


if __name__ == "__main__":
    check([copy_back_and_forth] * 10, [0, 1], 100)

Error (also happens in the other direction, write after read):

WARNING: ThreadSanitizer: data race (pid=192991)
  Atomic read of size 8 at 0x7f2d7a2b6260 by thread T10:
    #0 __tsan_atomic64_load ../../../../src/libsanitizer/tsan/tsan_interface_atomic.cpp:539 (libtsan.so.0+0x7fe0e)
    #1 _Py_atomic_load_ptr Include/cpython/pyatomic_gcc.h:300 (python+0x20434f)
    #2 _Py_TryXGetRef Include/internal/pycore_object.h:649 (python+0x20434f)
    #3 list_get_item_ref Objects/listobject.c:364 (python+0x20434f)
    #4 _PyList_GetItemRef Objects/listobject.c:415 (python+0x20a568)
    #5 _PyEval_EvalFrameDefault Python/generated_cases.c.h:686 (python+0x41e893)
    #6 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x46da1b)
    #7 _PyEval_Vector Python/ceval.c:1820 (python+0x46da1b)
    #8 _PyFunction_Vectorcall Objects/call.c:413 (python+0x188ec4)
    #9 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x190b3c)
    #10 method_vectorcall Objects/classobject.c:72 (python+0x190b3c)
    #11 _PyVectorcall_Call Objects/call.c:273 (python+0x18c9b7)
    #12 _PyObject_Call Objects/call.c:348 (python+0x18ceaf)
    #13 PyObject_Call Objects/call.c:373 (python+0x18cf34)
    #14 thread_run Modules/_threadmodule.c:354 (python+0x6689f2)
    #15 pythread_wrapper Python/thread_pthread.h:242 (python+0x57d03b)

  Previous write of size 8 at 0x7f2d7a2b6260 by thread T2:
    #0 PyList_SET_ITEM Include/cpython/listobject.h:47 (python+0x40aa3c)
    #1 _PyEval_EvalFrameDefault Python/generated_cases.c.h:11260 (python+0x466be8)
    #2 _PyEval_EvalFrame Include/internal/pycore_ceval.h:116 (python+0x46da1b)
    #3 _PyEval_Vector Python/ceval.c:1820 (python+0x46da1b)
    #4 _PyFunction_Vectorcall Objects/call.c:413 (python+0x188ec4)
    #5 _PyObject_VectorcallTstate Include/internal/pycore_call.h:167 (python+0x190b3c)
    #6 method_vectorcall Objects/classobject.c:72 (python+0x190b3c)
    #7 _PyVectorcall_Call Objects/call.c:273 (python+0x18c9b7)
    #8 _PyObject_Call Objects/call.c:348 (python+0x18ceaf)
    #9 PyObject_Call Objects/call.c:373 (python+0x18cf34)
    #10 thread_run Modules/_threadmodule.c:354 (python+0x6689f2)
    #11 pythread_wrapper Python/thread_pthread.h:242 (python+0x57d03b)

Relevant locations in files:

Write:

cpython/Python/generated_cases.c.h

Line 11262 in 052cb71

PyList_SET_ITEM(list, index, PyStackRef_AsPyObjectSteal(value));

Read:

cpython/Python/generated_cases.c.h

Line 686 in 052cb71

PyObject *res_o = _PyList_GetItemRef((PyListObject*)list, index);

What is happening probably is that the read is a lock-free read which occurs in the other thread between the PyList_SET_ITEM() and the UNLOCK_OBJECT(). In order to avoid TSAN complaining here either the PyList_SET_ITEM() and the read would have to be atomic, or the read would have to be locked with a mutex (negating the point of lock-free).

The thing is as far as I see the worst thing that will happen on the read side is it will get a stale value from the list. Which since there is not a defined order between the threads is harmless since the read can easily have happened before the modification or after regardless. So is this an issue and should the write (and read) be made atomic during lock-free operation (or some other correction applied)?

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

• gh-130920: Fix data race in STORE_SUBSCR_LIST_INT #130923