Skip to content

Add "unstable" frame stack api #91371

Open
Open
Add "unstable" frame stack api#91371
Assignees
markshannon
Labels
3.11only security fixestopic-C-APItype-featureA feature request or enhancement

Description

@markshannon
markshannon
opened on Apr 4, 2022
BPO 47215
Nosy @markshannon, @brandtbucher
PRs
  • bpo-47215: Add undocumented, unstable FrameStack API for use by greenlets and similar libraries. #32303
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/markshannon'
closed_at = None
created_at = <Date 2022-04-04.11:59:07.553>
labels = ['expert-C-API', 'type-feature', '3.11']
title = 'Add "unstable" frame stack api'
updated_at = <Date 2022-04-04.12:09:47.075>
user = 'https://github.com/markshannon'

    bugs.python.org fields:

    activity = <Date 2022-04-04.12:09:47.075>
actor = 'Mark.Shannon'
assignee = 'Mark.Shannon'
closed = False
closed_date = None
closer = None
components = ['C API']
creation = <Date 2022-04-04.11:59:07.553>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 47215
keywords = ['patch']
message_count = 1.0
messages = ['416665']
nosy_count = 2.0
nosy_names = ['Mark.Shannon', 'brandtbucher']
pr_nums = ['32303']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue47215'
versions = ['Python 3.11']

    Linked PRs

    Activity

    1. markshannon commented on Apr 4, 2022

      @markshannon
      markshannon
      on Apr 4, 2022
      MemberAuthor

      We need to provide an API to create, swap and free frame stacks for greenlets.

      Since this is primarily for greenlets (and any other stackful coroutines libraries that want to use it) it will be "unstable".

      In this case, by "unstable" I mean:

      1. Starts with an underscore
      2. Gets PyAPI_FUNC annotations, so we don't strip the symbols from the executable
      3. Undocumented, except for comments that say it is unstable.

      The API will be:

      typedef struct _frame_stack {
    _PyStackChunk *current_chunk;
    PyObject **top;
    PyObject **limit;
    int chunk_size;
} _PyFrameStack;

PyAPI_FUNC(void) _PyFrameStack_Init(_PyFrameStack *fs, int chunk_size);
PyAPI_FUNC(void) _PyFrameStack_Swap(_PyFrameStack *fs);
PyAPI_FUNC(void) _PyFrameStack_Free(_PyFrameStack *fs);
    2. markshannon
      added
      3.11only security fixes
      on Apr 4, 2022
    3. markshannon
      self-assigned this
      on Apr 4, 2022
    4. markshannon
      added
      topic-C-API
      type-featureA feature request or enhancement
      3.11only security fixes
      on Apr 4, 2022
    5. markshannon
      self-assigned this
      on Apr 4, 2022
    6. markshannon
      added
      topic-C-API
      type-featureA feature request or enhancement
      on Apr 4, 2022
    7. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    8. vstinner commented on May 27, 2024

      @vstinner
      vstinner
      on May 27, 2024
      Member

      greenlet code to save the Python thread state (C++): https://github.com/python-greenlet/greenlet/blob/937f150e07823ee03344aeeb5111c0bb371a831d/src/greenlet/TPythonState.cpp#L111

      greenlet is used by eventlet and gevent projects.

      PyOdide has similar code (in C): https://github.com/pyodide/pyodide/blob/d317e66d174efa203c59f094eb7714c0b90f7613/src/core/stack_switching/pystate.c#L161-L187

    9. vstinner commented on May 27, 2024

      @vstinner
      vstinner
      on May 27, 2024
      Member

      cc @hoodmane

    10. vstinner
      added 3 commits that reference this issue on May 27, 2024

      WIP: pythongh-91371: Add PyFrameStack C API

      f495137

      WIP: pythongh-91371: Add PyFrameStack C API

      55eb346

      WIP: Draft PyFrameStack API

      8013b22

    11. vstinner commented on May 28, 2024

      @vstinner
      vstinner
      on May 28, 2024
      Member

      Experimental greenlet change extracting the "save/restore" logic into a C API: python-greenlet/greenlet#407

    12. vstinner commented on May 28, 2024

      @vstinner
      vstinner
      on May 28, 2024
      Member

      Members which should be saved/restored per Python version.

      Python 3.10 (4):

      • tstate->cframe
      • tstate->cframe->use_tracing
      • tstate->recursion_depth
      • tstate->trash_delete_nesting

      Python 3.11 (8):

      • tstate->cframe
      • tstate->cframe->current_frame
      • tstate->cframe->use_tracing
      • tstate->datastack_chunk
      • tstate->datastack_limit
      • tstate->datastack_top
      • tstate->recursion_remaining
      • tstate->trash_delete_nesting

      Python 3.12 (8):

      • tstate->c_recursion_remaining
      • tstate->cframe->current_frame
      • tstate->cframe
      • tstate->datastack_chunk
      • tstate->datastack_limit
      • tstate->datastack_top
      • tstate->py_recursion_remaining
      • tstate->trash.delete_nesting

      For Python 3.13, I'm not sure. I found 8 members:

      • tstate->c_recursion_remaining
      • tstate->context
      • tstate->current_frame (UPDATE)
      • tstate->datastack_chunk
      • tstate->datastack_limit
      • tstate->datastack_top
      • tstate->delete_later
      • tstate->py_recursion_remaining
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    3.11only security fixestopic-C-APItype-featureA feature request or enhancement

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions