From 0d1f779e6119f480b3416413cdc9fdd54e09e4da Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 12:51:00 -0600 Subject: [PATCH 1/8] Expand the definition of each allocator domain. --- Doc/c-api/memory.rst | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 94be7dde89dbf3..f36586372b8aa9 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -104,18 +104,26 @@ For example, :c:func:`PyMem_Free` must be used to free memory allocated using :c The three allocation domains are: -* Raw domain: intended for allocating memory for general-purpose memory - buffers where the allocation *must* go to the system allocator or where the - allocator can operate without an :term:`attached thread state`. The memory - is requested directly from the system. See :ref:`Raw Memory Interface `. - -* "Mem" domain: intended for allocating memory for Python buffers and - general-purpose memory buffers where the allocation must be performed with - an :term:`attached thread state`. The memory is taken from the Python private heap. +* Raw domain: intended for allocating memory independently + of the Python runtime. This includes cases where the + allocation *must* go to the system allocator, when the runtime might + not be initialized yet, or in non-Python threads. + It also includes where the allocator can operate without an :term:`attached thread state`. + Examples include general-purpose memory buffers, as well as global + state for applications that embed Python. + The memory is requested directly from the system. See :ref:`Raw Memory Interface `. + +* "Mem" domain: intended for allocating memory tied to the + current interpreter but *not* belonging to any Python objects. + Examples include Python buffers, *arrays* of objects, + general-purpose memory buffers associated with an extension module, + and where the allocation must be performed with an :term:`attached thread state`. + The memory is taken from the Python private heap. See :ref:`Memory Interface `. -* Object domain: intended for allocating memory for Python objects. The - memory is taken from the Python private heap. See :ref:`Object allocators `. +* Object domain: intended for allocating memory belonging to Python + objects (which are necessarily tied to the current interpreter). + The memory is taken from the Python private heap. See :ref:`Object allocators `. .. note:: From 822b85f60c2f8771bc4e9ba5d148b58b840a58f4 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 13:10:22 -0600 Subject: [PATCH 2/8] Clarify about the GIL. --- Doc/c-api/memory.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index f36586372b8aa9..62ddfbbc6a0858 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -108,21 +108,22 @@ The three allocation domains are: of the Python runtime. This includes cases where the allocation *must* go to the system allocator, when the runtime might not be initialized yet, or in non-Python threads. - It also includes where the allocator can operate without an :term:`attached thread state`. Examples include general-purpose memory buffers, as well as global state for applications that embed Python. + The allocator may be called with or without an :term:`attached thread state`. The memory is requested directly from the system. See :ref:`Raw Memory Interface `. * "Mem" domain: intended for allocating memory tied to the current interpreter but *not* belonging to any Python objects. - Examples include Python buffers, *arrays* of objects, - general-purpose memory buffers associated with an extension module, - and where the allocation must be performed with an :term:`attached thread state`. + Examples include Python buffers, *arrays* of objects, and + general-purpose memory buffers associated with an extension module. + The allocator will always be called with an :term:`attached thread state`. The memory is taken from the Python private heap. See :ref:`Memory Interface `. * Object domain: intended for allocating memory belonging to Python objects (which are necessarily tied to the current interpreter). + The allocator will always be called with an :term:`attached thread state`. The memory is taken from the Python private heap. See :ref:`Object allocators `. .. note:: From cff9e075c655ce48f2d5a3733c757b92e1480878 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 13:56:54 -0600 Subject: [PATCH 3/8] Add references to the interface/custom sections. --- Doc/c-api/memory.rst | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 62ddfbbc6a0858..b22b180aa5e162 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -111,7 +111,9 @@ The three allocation domains are: Examples include general-purpose memory buffers, as well as global state for applications that embed Python. The allocator may be called with or without an :term:`attached thread state`. - The memory is requested directly from the system. See :ref:`Raw Memory Interface `. + The memory is requested directly from the system. + (Also see: :ref:`user API `, + :ref:`custom allocator `.) * "Mem" domain: intended for allocating memory tied to the current interpreter but *not* belonging to any Python objects. @@ -119,12 +121,15 @@ The three allocation domains are: general-purpose memory buffers associated with an extension module. The allocator will always be called with an :term:`attached thread state`. The memory is taken from the Python private heap. - See :ref:`Memory Interface `. + (Also see: :ref:`user API `, + :ref:`custom allocator `.) * Object domain: intended for allocating memory belonging to Python objects (which are necessarily tied to the current interpreter). The allocator will always be called with an :term:`attached thread state`. - The memory is taken from the Python private heap. See :ref:`Object allocators `. + The memory is taken from the Python private heap. + (Also see: :ref:`user API `, + :ref:`custom allocator `.) .. note:: @@ -140,6 +145,9 @@ The three allocation domains are: .. _raw-memoryinterface: + +.. _raw-interface: + Raw Memory Interface ==================== @@ -204,7 +212,7 @@ zero bytes. If *p* is ``NULL``, no operation is performed. -.. _memoryinterface: +.. _mem-interface: Memory Interface ================ @@ -343,6 +351,8 @@ They exist solely for backwards compatibility. .. _objectinterface: +.. _object-interface: + Object allocators ================= @@ -467,6 +477,13 @@ Legend: :option:`--with-hwaddress-sanitizer`, and/or :option:`--with-memory-sanitizer` option. + +.. _custom-raw-allocator: + +.. _custom-mem-allocator: + +.. _custom-object-allocator: + .. _customize-memory-allocators: Customize Memory Allocators From 1d6ee4338c4d9951993fb684eb660b844f1ee129 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 14:03:28 -0600 Subject: [PATCH 4/8] Clarify about using the same custom allocator to free memory. --- Doc/c-api/memory.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index b22b180aa5e162..122c2a7a865bcd 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -145,6 +145,11 @@ The three allocation domains are: .. _raw-memoryinterface: +Likewise, if one or more +:ref:`custom allocator ` is used +then any allocated memory must be freed using the same allocator +that provided it. + .. _raw-interface: From fa66d5ad2b59c2a6ac269a45a4c47cbf51109bcf Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 14:05:16 -0600 Subject: [PATCH 5/8] Be explicit about which allocator domain is used. --- Doc/c-api/memory.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 122c2a7a865bcd..6b982de38b30f6 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -160,7 +160,8 @@ The following function sets are wrappers to the system allocator. These functions are thread-safe, so a :term:`thread state` does not need to be :term:`attached `. -The :ref:`default raw memory allocator ` uses +These functions use the :ref:`raw ` allocator. +:ref:`By default `, the allocator uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`!free`; call ``malloc(1)`` (or ``calloc(1, 1)``) when requesting zero bytes. @@ -226,6 +227,7 @@ The following function sets, modeled after the ANSI C standard, but specifying behavior when requesting zero bytes, are available for allocating and releasing memory from the Python heap. +These functions use the :ref:`"mem" ` allocator. In the GIL-enabled build (default build) the :ref:`default memory allocator ` uses the :ref:`pymalloc memory allocator `, whereas in the @@ -371,6 +373,7 @@ memory from the Python heap. functions in this domain by the methods described in the :ref:`Customize Memory Allocators ` section. +These functions use the :ref:`"mem" ` allocator. The :ref:`default object allocator ` uses the :ref:`pymalloc memory allocator `. In the :term:`free-threaded ` build, the default is the From 0e1d17a644361d8b3ab8a627ec41a2dc0320a0ad Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 14:10:04 -0600 Subject: [PATCH 6/8] Clarify about using the same interpreter to free. --- Doc/c-api/memory.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 6b982de38b30f6..8015b5e51b4e72 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -160,6 +160,11 @@ The following function sets are wrappers to the system allocator. These functions are thread-safe, so a :term:`thread state` does not need to be :term:`attached `. +Use of these functions is not restricted by the current state of the +Python runtime. Likewise, they are not constrained by which interpreter +is active in the current thread or even if there is one. The allocated +memory is process-global and independent of the Python runtime. + These functions use the :ref:`raw ` allocator. :ref:`By default `, the allocator uses the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` @@ -227,6 +232,10 @@ The following function sets, modeled after the ANSI C standard, but specifying behavior when requesting zero bytes, are available for allocating and releasing memory from the Python heap. +All allocations happen relative to the current interpreter. Subsequent +free (and realloc) operations on the allocated memory must be made using +the same interpreter. + These functions use the :ref:`"mem" ` allocator. In the GIL-enabled build (default build) the :ref:`default memory allocator ` uses the @@ -373,6 +382,10 @@ memory from the Python heap. functions in this domain by the methods described in the :ref:`Customize Memory Allocators ` section. +All allocations happen relative to the current interpreter. Subsequent +free (and realloc) operations on the allocated memory must be made using +the same interpreter. + These functions use the :ref:`"mem" ` allocator. The :ref:`default object allocator ` uses the :ref:`pymalloc memory allocator `. In the From e21659f98860820c03da7e68451b461145a6a230 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 15:54:09 -0600 Subject: [PATCH 7/8] Add an explanation to the custom allocators section. --- Doc/c-api/memory.rst | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 8015b5e51b4e72..891079a89b2e47 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -510,6 +510,35 @@ Legend: Customize Memory Allocators =========================== +The Python runtime may be configured, with :c:func:`PyMem_SetAllocator`, +to use custom allocators for any of the +:ref:`supported allocator domains `. +There are two kinds of custom allocator: + +* actual allocator implementations +* wrappers around other allocators (AKA "hooks") + +Applications that embed Python may set either kind before the runtime +is initialized (e.g. with :c:func:`Py_InitializeFromConfig`). However, +from that point on only wrappers around the current allocator +(see :c:func:`PyMem_GetAllocator`) may be set. Thus, extension modules +may only set wrappers. + +An actual allocator is responsible for managing its own state and memory +pool. For the "mem" and "object" domains, the allocator is responsible +for maintaining its state and allocations relative to the +:c:func:`current interpreter `. +The ``PyMemAllocatorEx.ctx`` field may be ``NULL`` for +this kind of allocator. + +A wrapper can be useful for tracking allocations, adjusting the behavior +of another allocator (e.g. handling failures differently), or debugging. +For example, debug builds of CPython use a wrapper to track memory usage +and find memory leaks, and the :mod:`tracemalloc` module sets a wrapper +to identify where allocations happen. Typically, the wrapped allocator +is stored in (or within) the ``PyMemAllocatorEx.ctx`` field +of the wrapper. + .. versionadded:: 3.4 .. c:type:: PyMemAllocatorEx From 130ffa92edb174a040e59e936a9665e0471dd3f7 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Wed, 6 Sep 2023 15:54:58 -0600 Subject: [PATCH 8/8] Enumerate characteristics for custom allocators. --- Doc/c-api/memory.rst | 50 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 44 insertions(+), 6 deletions(-) diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst index 891079a89b2e47..16d47960787537 100644 --- a/Doc/c-api/memory.rst +++ b/Doc/c-api/memory.rst @@ -499,12 +499,6 @@ Legend: :option:`--with-memory-sanitizer` option. -.. _custom-raw-allocator: - -.. _custom-mem-allocator: - -.. _custom-object-allocator: - .. _customize-memory-allocators: Customize Memory Allocators @@ -539,6 +533,50 @@ to identify where allocations happen. Typically, the wrapped allocator is stored in (or within) the ``PyMemAllocatorEx.ctx`` field of the wrapper. +Regardless of the kind, the following applies to every custom +allocator, depending on the domain: + +.. _custom-raw-allocator: + +Raw domain: + +* a system allocator (or wrapper around one) +* the memory is requested directly from the system +* independent of the Python runtime +* not associated with any interpreter +* may be used before the runtime is initialized and after it's finalized +* may be used with the runtime in an unknown state +* must be thread-safe +* must work whether or not the :term:`GIL` is held + +.. _custom-mem-allocator: + +"Mem" domain: + +* may use a Python-specific heap +* allocated memory will be used only in a single interpreter +* memory for each interpreter is likely kept separate +* must be thread-safe +* always called with the runtime in an active, stable state +* always called with the target interpreter/tstate set in the current thread + (:c:func:`PyInterpreterState_Get`, :c:func:`PyThreadState_Get`) +* always called with the current interpreter's :term:`GIL` held +* never used for memory belonging to Python objects + +.. _custom-object-allocator: + +Object domain: + +* will use a Python-specific heap +* allocated memory will be used only in a single interpreter +* memory for each interpreter is kept separate +* must be thread-safe +* always called with the runtime in an active, stable state +* always called with the target interpreter/tstate set in the current thread + (:c:func:`PyInterpreterState_Get`, :c:func:`PyThreadState_Get`) +* always called with the current interpreter's :term:`GIL` held +* only used for memory belonging to Python objects + .. versionadded:: 3.4 .. c:type:: PyMemAllocatorEx