Skip to content

Race between _PyObject_GenericGetAttrWithDict and ensure_nonmanaged_dict under free-threading #128100

Closed
Closed
Race between _PyObject_GenericGetAttrWithDict and ensure_nonmanaged_dict under free-threading#128100
Labels
3.13only security fixes3.14bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or error

Description

@hawkinsp
hawkinsp
opened on Dec 19, 2024

Bug report

Bug description:

Under Python 3.13.1t build with thread-sanitizer:

import concurrent.futures
import functools
import threading

num_threads = 100


def closure(b, o):
  b.wait()
  getattr(o, "foo", None)
  o.foo = 42

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(100):
    b = threading.Barrier(num_threads)
    o = functools.partial(lambda x: x, 42)
    for _ in range(num_threads):
      executor.submit(functools.partial(closure, b, o))

Get (it may take a few runs):

WARNING: ThreadSanitizer: data race (pid=239136)
  Read of size 8 at 0x7fa5e69e6a28 by thread T91:
    #0 _PyObject_GenericGetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1665:24 (python3.13+0x2960a5) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 PyObject_GetOptionalAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1266:19 (python3.13+0x2930c4) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #2 builtin_getattr /usr/local/google/home/phawkins/p/cpython/Python/bltinmodule.c:1200:13 (python3.13+0x3d8bf9) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 cfunction_vectorcall_FASTCALL /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:425:24 (python3.13+0x289efb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #4 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous atomic write of size 8 at 0x7fa5e69e6a28 by thread T18:
    #0 _Py_atomic_store_ptr_release /usr/local/google/home/phawkins/p/cpython/./Include/cpython/pyatomic_gcc.h:501:3 (python3.13+0x272a17) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #1 ensure_nonmanaged_dict /usr/local/google/home/phawkins/p/cpython/Objects/dictobject.c:7275:9 (python3.13+0x272a17)
    #2 _PyObjectDict_SetItem /usr/local/google/home/phawkins/p/cpython/Objects/dictobject.c:7311:12 (python3.13+0x272aee) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #3 _PyObject_GenericSetAttrWithDict /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1791:19 (python3.13+0x297c8e) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #4 PyObject_GenericSetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1819:12 (python3.13+0x298667) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #5 PyObject_SetAttr /usr/local/google/home/phawkins/p/cpython/Objects/object.c:1385:15 (python3.13+0x294f8a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:5488:27 (python3.13+0x3f3f6d) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #8 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #9 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x571bb2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 partial_vectorcall /usr/local/google/home/phawkins/p/cpython/./Modules/_functoolsmodule.c:252:16 (python3.13+0x571bb2)
    #12 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #14 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #15 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #16 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #18 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #21 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #23 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #24 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

Activity

  1. hawkinsp
    added
    type-bugAn unexpected behavior, bug, or error
    on Dec 19, 2024
  2. ZeroIntensity
    added
    3.13only security fixes
    topic-free-threading
    3.14bugs and security fixes
    on Dec 19, 2024

  3. colesbury commented on Dec 19, 2024

    @colesbury
    colesbury
    on Dec 19, 2024
    Contributor

    I think there are two data races related to this:

    1. The dictionary load in _PyObject_GenericGetAttrWithDict should be atomic (with acquire ordering since we're not using consume)
    2. We need to handle the case where you change the dictionary (via obj.__dict__ = newdict). That doesn't show up in this example, but it's relevant since it may also race with the load. That was partly addressed in the following PR, but I don't think PyObject_GenericSetDict got updated.

  4. wrongnull commented on Dec 26, 2024

    @wrongnull
    wrongnull
    on Dec 26, 2024
    Contributor

    I think it is worth adding _PyObject_GenericGetAttrWithDict here #117657 since it is reproducible in the manner described in this issue

  5. colesbury
    added a commit that references this issue on Dec 30, 2024

    gh-128100: Use atomic dictionary load in `_PyObject_GenericGetAttrWit…

    47d2cb8
  6. srinivasreddy
    added a commit that references this issue on Jan 8, 2025

    pythongh-128100: Use atomic dictionary load in `_PyObject_GenericGetA…

    b5cc4cf
  7. miss-islington
    added a commit that references this issue on Feb 10, 2025

    pythongh-128100: Use atomic dictionary load in `_PyObject_GenericGetA…

    0cf631d
  8. colesbury
    added a commit that references this issue on Feb 10, 2025

    [3.13] gh-128100: Use atomic dictionary load in `_PyObject_GenericGet…

    0526322

  9. ZeroIntensity commented on Feb 17, 2025

    @ZeroIntensity
    ZeroIntensity
    on Feb 17, 2025
    Member

    Thanks @hawkinsp for the report and @wrongnull for the fix!

  10. ZeroIntensity
    closed this as completedon Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixestopic-free-threadingtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions