Skip to content

The datetime C API is not thread-safe #141563

Closed
#144830
Closed
The datetime C API is not thread-safe#141563
#144830
Labels
3.15bugs and security fixesextension-modulesC modules in the Modules dirtopic-C-APItopic-free-threadingtopic-subinterpreterstype-bugAn unexpected behavior, bug, or error

Description

@ZeroIntensity
ZeroIntensity
opened on Nov 14, 2025

Bug report

Bug description:

datetime has a C API that we expose through a global variable. This does not work well under multiple threads. For example, if PyDateTime_IMPORT fails in one thread while another thread is invoking the datetime C API, then that will set PyDateTimeAPI to NULL, leading to NULL pointer dereferences in the original thread.

This global variable also forces us to do some nasty hacks with subinterpreters because it exposes static types. I helped fix this a few months ago by making _datetime static (see this thread), but there are probably other issues that I haven't found yet.

I think a bandaid fix would be to make PyDateTimeAPI a thread-local variable, but I suspect that would break a lot of things, nor would that fix this for subinterpreters. More generally, we could deprecate PyDateTime_IMPORT and PyDateTimeAPI in favor of letting users manage the pointer on their own, but I don't know if we can do this without too many code changes (see #141543 and #83785 for the docs side).

cc @pganssle

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs

Activity

  1. ZeroIntensity
    added
    type-bugAn unexpected behavior, bug, or error
    extension-modulesC modules in the Modules dir
    topic-subinterpreters
    topic-C-API
    on Nov 14, 2025
  2. github-project-automation
    added this to Subinterpreterson Nov 14, 2025
  3. ZeroIntensity
    added
    3.13only security fixes
    topic-free-threading
    3.14bugs and security fixes
    3.15bugs and security fixes
    on Nov 14, 2025
  4. github-project-automation
    moved this to Todo in Subinterpreterson Nov 14, 2025
  5. StanFromIreland
    added this to Date and time issues 🕰️on Nov 14, 2025

  6. sharktide commented on Nov 15, 2025

    @sharktide
    sharktide
    on Nov 15, 2025
    Contributor

    Has anyone found any examples of this on GitHub? I imagine that this might happen because it seems like it would be common to use dateline to measure the time of something executing but at the same time why use datetime itself? There has to be better libs or inbuilt functions for most use cases

  7. ZeroIntensity commented on Nov 15, 2025

    @ZeroIntensity
    ZeroIntensity
    on Nov 15, 2025
    MemberAuthor

    Yes.

  8. kumaraditya303 commented on Jan 21, 2026

    @kumaraditya303
    kumaraditya303
    on Jan 21, 2026
    Contributor

    I think we can make it thread safe with atomics, basically create a exchange loop and only initialize it if it is not initialized already.

    Something like this should work:

    diff --git a/Include/datetime.h b/Include/datetime.h
index ed36e6e48c8..17c85b19e11 100644
--- a/Include/datetime.h
+++ b/Include/datetime.h
@@ -197,7 +197,15 @@ typedef struct {
 static PyDateTime_CAPI *PyDateTimeAPI = NULL;
 
 #define PyDateTime_IMPORT \
-    PyDateTimeAPI = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0)
+    do
+    {
+        PyDateTime_CAPI *capi = (PyDateTime_CAPI *)PyCapsule_Import(PyDateTime_CAPSULE_NAME, 0); \
+        if (capi && _Py_atomic_load_ptr(&PyDateTimeAPI) == NULL) { \
+            PyDateTime_CAPI *expected_api = NULL; \
+            _Py_atomic_compare_exchange_ptr(&PyDateTimeAPI, &expected_api, capi); \
+        }
+    } while (0)
+
 
 /* Macro for access to the UTC singleton */
 #define PyDateTime_TimeZone_UTC PyDateTimeAPI->TimeZone_UTC

  9. ZeroIntensity commented on Jan 21, 2026

    @ZeroIntensity
    ZeroIntensity
    on Jan 21, 2026
    MemberAuthor

    I think that will probably break code that relies on PyDateTimeAPI for checking if PyDateTime_IMPORT failed, so that's probably too much for a patch release. Maybe it'll work for 3.15 though?

  10. 26 remaining items

  11. thunder-coding
    added 3 commits that reference this issue on Feb 15, 2026

    pythongh-141563: Fix test_cext on Windows (python#144677)

    6fcabbf

    pythongh-141563: Fix test_cppext on macOS (python#144685)

    5ffc128

    pythongh-141563: Enable test_cppext internal C API tests on macOS (py…

    d2bfb17
  12. kumaraditya303
    closed this as completedin #144830on Feb 15, 2026
  13. brijkapadia
    added 5 commits that reference this issue on Feb 28, 2026

    pythongh-141563: Add missing cast to _PyDateTime_IMPORT() (python#144667

    9784156

    pythongh-141563: Don't test datetime.h with the limited C API (python…

    fceed06

    pythongh-141563: Fix test_cext on Windows (python#144677)

    227e8f4

    pythongh-141563: Fix test_cppext on macOS (python#144685)

    f1dcc5b

    pythongh-141563: Enable test_cppext internal C API tests on macOS (py…

    974cbd7
  14. ljfp
    added 6 commits that reference this issue on Apr 25, 2026

    pythongh-141563: make `PyDateTime_IMPORT` thread-safe (python#144210)

    9d08faa

    pythongh-141563: Add missing cast to _PyDateTime_IMPORT() (python#144667

    77e2b81

    pythongh-141563: Don't test datetime.h with the limited C API (python…

    46e8e0e

    pythongh-141563: Fix test_cext on Windows (python#144677)

    544b200

    pythongh-141563: Fix test_cppext on macOS (python#144685)

    6af991d

    pythongh-141563: Enable test_cppext internal C API tests on macOS (py…

    74fa82e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.15bugs and security fixesextension-modulesC modules in the Modules dirtopic-C-APItopic-free-threadingtopic-subinterpreterstype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions