Skip to content

The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing) #87926

Closed
Closed
The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)#87926
Labels
3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)pendingThe issue will be closed if no feedback is providedperformancePerformance or resource usagetopic-C-API

Description

@markshannon
markshannon
opened on Apr 7, 2021
BPO 43760
Nosy @gvanrossum, @rhettinger, @scoder, @vstinner, @encukou, @ambv, @markshannon, @hroncok, @pablogsal, @miss-islington, @erlend-aasland, @Xtrem532
PRs
  • bpo-43760: Streamline dispatch sequence for machines without computed gotos. #25244
  • bpo-43760: Speed up check for tracing in interpreter dispatch #25276
  • [3.10] bpo-43760: Ensure that older Cython generated code compiles under 3.10. #28474
  • [3.10] bpo-43760: Ensure that older Cython generated code compiles under 3.10 #28498
  • bpo-43760: Document PyThreadState.use_tracing removal #28527
  • [3.10] bpo-43760: Document PyThreadState.use_tracing removal (GH-28527) #28529
  • bpo-43760: Add PyThreadState_EnterTracing() #28542
  • bpo-43760: Check for tracing using 'bitwise or' instead of branch in dispatch. #28723
  • bpo-43760: Rename _PyThreadState_DisableTracing() #29032
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2021-04-07.09:35:49.770>
labels = ['interpreter-core', 'expert-C-API', '3.10', 'performance']
title = 'The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)'
updated_at = <Date 2021-11-08.17:16:37.213>
user = 'https://github.com/markshannon'

    bugs.python.org fields:

    activity = <Date 2021-11-08.17:16:37.213>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core', 'C API']
creation = <Date 2021-04-07.09:35:49.770>
creator = 'Mark.Shannon'
dependencies = []
files = []
hgrepos = []
issue_num = 43760
keywords = ['patch']
message_count = 46.0
messages = ['390410', '390522', '390951', '393379', '393384', '393385', '393389', '393390', '393401', '393403', '393404', '393407', '393410', '393425', '393459', '393466', '393666', '402150', '402162', '402216', '402225', '402231', '402234', '402235', '402358', '402360', '402362', '402415', '402416', '402426', '402432', '402434', '402481', '402496', '402525', '402594', '402603', '403166', '403217', '404020', '404024', '404025', '404173', '404197', '404600', '405968']
nosy_count = 13.0
nosy_names = ['gvanrossum', 'rhettinger', 'jpe', 'scoder', 'vstinner', 'petr.viktorin', 'lukasz.langa', 'Mark.Shannon', 'hroncok', 'pablogsal', 'miss-islington', 'erlendaasland', 'Xtrem532']
pr_nums = ['25244', '25276', '28474', '28498', '28527', '28529', '28542', '28723', '29032']
priority = None
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue43760'
versions = ['Python 3.10']

    Activity

    1. markshannon commented on Apr 7, 2021

      @markshannon
      markshannon
      on Apr 7, 2021
      MemberAuthor

      The DISPATCH() macro has two failings.

      1. Its check for tracing involves too much pointer chaser.

      2. The logic assumes that computed-gotos is the "fast path" which makes switch dispatch, and therefore Python on Windows unnecessarily slow.

    2. markshannon commented on Apr 8, 2021

      @markshannon
      markshannon
      on Apr 8, 2021
      MemberAuthor

      New changeset 28d28e0 by Mark Shannon in branch 'master':
      bpo-43760: Streamline dispatch sequence for machines without computed gotos. (GH-25244)
      28d28e0

    3. markshannon commented on Apr 13, 2021

      @markshannon
      markshannon
      on Apr 13, 2021
      MemberAuthor

      New changeset 9e7b207 by Mark Shannon in branch 'master':
      bpo-43760: Speed up check for tracing in interpreter dispatch (bpo-25276)
      9e7b207

    4. hroncok commented on May 10, 2021

      @hroncok
      hroncokmannequin
      on May 10, 2021
      Mannequin

      I am afraid the "Speed up check for tracing in interpreter dispatch" brought some backwards incompatible changes:

      yappi/_yappi.c:1261:9: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘use_tracing’; did you mean ‘tracing’?
      1261 | ts->use_tracing = 1;
      | ^~~~~~~~~~~
      | tracing

      This is not mentioned in https://docs.python.org/3.10/whatsnew/3.10.html and I haven't noticed the use_tracing member being deprecated. I am confused. Should this happened?

    5. hroncok commented on May 10, 2021

      @hroncok
      hroncokmannequin
      on May 10, 2021
      Mannequin

      Fedora packages affected (that we know of now):

      greenlet: https://bugzilla.redhat.com/show_bug.cgi?id=1957784
      dipy: https://bugzilla.redhat.com/show_bug.cgi?id=1958203
      yappi: https://bugzilla.redhat.com/show_bug.cgi?id=1958896
      smartcols: https://bugzilla.redhat.com/show_bug.cgi?id=1958938

    6. markshannon commented on May 10, 2021

      @markshannon
      markshannon
      on May 10, 2021
      MemberAuthor

      At yappi/_yappi.c:1261 sets an undocumented field on a CPython internal data structure.

      What did you believe that was supposed to do? use_tracing is not documented anywhere.

      We could add the field back and ignore it, but I doubt that would help you much.

    7. markshannon commented on May 10, 2021

      @markshannon
      markshannon
      on May 10, 2021
      MemberAuthor

      If there is no C-API function that supports your needs, feel free to suggest one.

    8. hroncok commented on May 10, 2021

      @hroncok
      hroncokmannequin
      on May 10, 2021
      Mannequin

      Disclaimer: I have not written the code nor do I understand what is trying to achieve. I merely collect the data and report the problems to the package maintainers.

      It just seems to me that a non-underscored (and hence public) member variable on a non-underscored (and hence public) structure should not suddenly go missing. Although, I am not familiar with the rules that define what part of the API falls under https://www.python.org/dev/peps/pep-0497/

    9. hroncok commented on May 10, 2021

      @hroncok
      hroncokmannequin
      on May 10, 2021
      Mannequin

      scikit-learn: https://bugzilla.redhat.com/show_bug.cgi?id=1958976

      gcc: sklearn/cluster/_k_means_fast.c
      In file included from /usr/lib64/python3.10/site-packages/numpy/core/include/numpy/ndarraytypes.h:1944,
      from /usr/lib64/python3.10/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
      from /usr/lib64/python3.10/site-packages/numpy/core/include/numpy/arrayobject.h:4,
      from sklearn/cluster/_k_means_fast.c:635:
      /usr/lib64/python3.10/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
      17 | #warning "Using deprecated NumPy API, disable it with " \
      | ^~~~~~~
      sklearn/cluster/_k_means_fast.c: In function ‘__Pyx_call_return_trace_func’:
      sklearn/cluster/_k_means_fast.c:1596:15: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘use_tracing’; did you mean ‘tracing’?
      1596 | tstate->use_tracing = 0;
      | ^~~~~~~~~~~
      | tracing
      sklearn/cluster/_k_means_fast.c:1602:15: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘use_tracing’; did you mean ‘tracing’?
      1602 | tstate->use_tracing = 1;
      | ^~~~~~~~~~~
      | tracing

      The usage comes from https://github.com/cython/cython/blob/master/Cython/Utility/Profile.c

    10. encukou commented on May 10, 2021

      @encukou
      encukou
      on May 10, 2021
      Member

      PEP-0497 is rejected; the active one is PEP-387, which says "backwards incompatibility" means preexisting code ceases to comparatively function after a change.
      So, this does look like a backwards-incompatible change.

      Unfortunately, not all of the C API is documented, so unless it's explicitly marked private, people will use it :(

    11. markshannon commented on May 10, 2021

      @markshannon
      markshannon
      on May 10, 2021
      MemberAuthor

      But what does "use it" mean?
      What does setting tstate->use_tracing = 1 do?
      There is no documented behavior, so how do we know what assumptions people are making about what happens when they set some field to 1?

      As I said, we could keep the field and ignore it, but that seems worse.

    12. rhettinger commented on May 10, 2021

      @rhettinger
      rhettinger
      on May 10, 2021
      Contributor

      I don't think the PEP meant to restrict individual struct member such as this. For example, we were able to switch from byte code to word code without violating the intended rules. Consider asking Brett and Benjamin for clarification. I would think that if a new function were introduced to provide a reliable way to determine whether tracing was enabled, that would suffice for external packages to have a minimally disruptive migration path.

    13. vstinner commented on May 10, 2021

      @vstinner
      vstinner
      on May 10, 2021
      Member

      I understand that some projects manually call the profile and/or trace functions, and temporarily set use_tracing 0 while calling these functions.

      Some projects restore use_tracing to the correct value (compute the efficient value), some projects simply set use_tracing to 1.

      I see 3 use cases:

      • disable tracing temporarily (set use_tracing to 0)
      • reenable tracing (compute use_tracing to the correct value)
      • check if tracing is used (get use_tracing)

      We can add 3 functions:

      • PyThreadState_DisableTracing()
      • PyThreadState_EnableTracing()
      • PyThreadState_GetTracing()

      PyThreadState_EnableTracing(tstate) would do something like:

      tstate->cframe->use_tracing = (tstate->c_tracefunc || tstate->c_profilefunc);

      If we added these functions, I can then add an implementation for Python 3.9 and older to my https://github.com/pythoncapi/pythoncapi_compat project for backward compatibility.

      The problem is that some projects also increase temporarily ts->tracing. Since I would like to make PyThreadState opaque, I would prefer to hide this access behind a function call as well. Maybe we need an API to call profile and/or trace functions?

      --

      According to the bugzilla compiler errors:

      greenlet: https://bugzilla.redhat.com/show_bug.cgi?id=1957784

      It has already been fixed:

      It uses:

      • "tstate->use_tracing = 0;"
      • "tstate->use_tracing = (tstate->tracing <= 0 && (...)"

      dipy: https://bugzilla.redhat.com/show_bug.cgi?id=1958203

      It uses:

      • "tstate->use_tracing = 0;"
      • "tstate->use_tracing = 1;"
      • "tstate->use_tracing = (tstate->c_profilefunc || (...)"
      • "return tstate->use_tracing && retval;"
      • "if (tstate->use_tracing) {"

      yappi: https://bugzilla.redhat.com/show_bug.cgi?id=1958896

      • "ts->use_tracing = 1;"
      • "ts->use_tracing = 0;"

      smartcols: https://bugzilla.redhat.com/show_bug.cgi?id=1958938

      It uses "tstate->use_tracing = 0;".

      scikit-learn: https://bugzilla.redhat.com/show_bug.cgi?id=1958976

      It uses:

      • "tstate->use_tracing = 0;"
      • "tstate->use_tracing = 1;"

      The usage comes from https://github.com/cython/cython/blob/master/Cython/Utility/Profile.c

      Simplified code:

      --------------

      static int __Pyx_TraceSetupAndCall(...)
{
    ...
    tstate->tracing++;
    tstate->use_tracing = 0;
      if (tstate->c_tracefunc)
    retval = tstate->c_tracefunc(tstate->c_traceobj, *frame, PyTrace_CALL, NULL) == 0;
if (retval && tstate->c_profilefunc)
    retval = tstate->c_profilefunc(tstate->c_profileobj, *frame, PyTrace_CALL, NULL) == 0;

tstate-\>use_tracing = (tstate-\>c_profilefunc ||
                       (CYTHON_TRACE && tstate-\>c_tracefunc));
tstate-\>tracing--;
...

      }

        int __Pyx_use_tracing = 0;

  #define __Pyx_TraceCall(funcname, srcfile, firstlineno, nogil, goto_error)             \
  if (nogil) {                                                                           \
      if (CYTHON_TRACE_NOGIL) {                                                          \
          PyThreadState *tstate;                                                         \
          PyGILState_STATE state = PyGILState_Ensure();                                  \
          tstate = __Pyx_PyThreadState_Current;                                          \
          if (unlikely(tstate->use_tracing) && !tstate->tracing &&                       \
                  (tstate->c_profilefunc || (CYTHON_TRACE && tstate->c_tracefunc))) {    \
              __Pyx_use_tracing = __Pyx_TraceSetupAndCall(&$frame_code_cname, &$frame_cname, tstate, funcname, srcfile, firstlineno);  \
          }                                                                              \
          PyGILState_Release(state);                                                     \
          if (unlikely(__Pyx_use_tracing < 0)) goto_error;                               \
      }                                                                                  \
  } else {                                                                               \
      PyThreadState* tstate = PyThreadState_GET();                                       \
      if (unlikely(tstate->use_tracing) && !tstate->tracing &&                           \
              (tstate->c_profilefunc || (CYTHON_TRACE && tstate->c_tracefunc))) {        \
          __Pyx_use_tracing = __Pyx_TraceSetupAndCall(&$frame_code_cname, &$frame_cname, tstate, funcname, srcfile, firstlineno);  \
          if (unlikely(__Pyx_use_tracing < 0)) goto_error;                               \
      }                                                                                  \
  }
    14. vstinner
      added
      3.10 (EOL)end of life
      on May 10, 2021
    15. vstinner
      changed the title [-]The DISPATCH() macro is not as efficient as it could be.[/-] [+]The DISPATCH() macro is not as efficient as it could be (move PyThreadState.use_tracing)[/+] on May 10, 2021

    16. 30 remaining items

    17. gvanrossum commented on Sep 25, 2021

      @gvanrossum
      gvanrossum
      on Sep 25, 2021
      Member

      Ah, I think the docs need to be clarified a bit. Here's what I was missing:

      The key thing to know here is that there are three state variables; c_tracefunc, c_profilefunc (on the thread state), and use_tracing (on the C frame).

      Normally use_tracing is initialized to false if both functions are NULL, and true otherwise (if at least one of the functions is set).

      Disabling means setting use_tracing to false regardless. Resetting means setting use_tracing to the value computed above.

      There's also a fourth variable, tstate->tracing, which indicates whether a tracing function is active (i.e., it has been called and hasn't exited yet). This can be incremented and decremented. But none of the proposed APIs affect it.

      Would it be reasonable to just put these APIs in pythoncapi_compat, instead of in the stdlib? (It would be yet one more selling point for people to start using that. :-)

    18. pablogsal commented on Oct 4, 2021

      @pablogsal
      pablogsal
      on Oct 4, 2021
      Member

      New changeset 78184fa by Pablo Galindo (Miss Islington (bot)) in branch '3.10':
      bpo-43760: Document PyThreadState.use_tracing removal (GH-28527) (GH-28529)
      78184fa

    19. markshannon commented on Oct 5, 2021

      @markshannon
      markshannon
      on Oct 5, 2021
      MemberAuthor

      New changeset bd627eb by Mark Shannon in branch 'main':
      bpo-43760: Check for tracing using 'bitwise or' instead of branch in dispatch. (GH-28723)
      bd627eb

    20. vstinner commented on Oct 15, 2021

      @vstinner
      vstinner
      on Oct 15, 2021
      Member

      New changeset 547d26a by Victor Stinner in branch 'main':
      bpo-43760: Add PyThreadState_EnterTracing() (GH-28542)
      547d26a

    21. vstinner commented on Oct 15, 2021

      @vstinner
      vstinner
      on Oct 15, 2021
      Member

      bpo-43760: Add PyThreadState_EnterTracing() (GH-28542)

      I created changes to use it:

    22. vstinner commented on Oct 15, 2021

      @vstinner
      vstinner
      on Oct 15, 2021
      Member

      PyThreadState.cframe.use_tracing format changed again: set value set to 0 or 255.
      bd627eb

    23. vstinner commented on Oct 18, 2021

      @vstinner
      vstinner
      on Oct 18, 2021
      Member

      Merged:

    24. vstinner commented on Oct 18, 2021

      @vstinner
      vstinner
      on Oct 18, 2021
      Member

      New changeset 034f607 by Victor Stinner in branch 'main':
      bpo-43760: Rename _PyThreadState_DisableTracing() (GH-29032)
      034f607

    25. vstinner commented on Oct 21, 2021

      @vstinner
      vstinner
      on Oct 21, 2021
      Member

      I created #29121 to add PyThreadState_SetProfile() and PyThreadState_SetTrace() functions.

    26. vstinner commented on Nov 8, 2021

      @vstinner
      vstinner
      on Nov 8, 2021
      Member

      greenlet now uses PyThreadState_EnterTracing() and PyThreadState_LeaveTracing() rather than accessing directly use_tracing:

      python-greenlet/greenlet@9b49da5

      On Python 3.10, it implements these functions with:
      ---

      // bpo-43760 added PyThreadState_EnterTracing() to Python 3.11.0a2
#if PY_VERSION_HEX < 0x030B00A2 && !defined(PYPY_VERSION)
static inline void PyThreadState_EnterTracing(PyThreadState *tstate)
{
    tstate->tracing++;
#if PY_VERSION_HEX >= 0x030A00A1
    tstate->cframe->use_tracing = 0;
#else
    tstate->use_tracing = 0;
#endif
}
#endif

// bpo-43760 added PyThreadState_LeaveTracing() to Python 3.11.0a2
#if PY_VERSION_HEX < 0x030B00A2 && !defined(PYPY_VERSION)
static inline void PyThreadState_LeaveTracing(PyThreadState *tstate)
{
    tstate->tracing--;
    int use_tracing = (tstate->c_tracefunc != NULL
                       || tstate->c_profilefunc != NULL);
#if PY_VERSION_HEX >= 0x030A00A1
    tstate->cframe->use_tracing = use_tracing;
#else
    tstate->use_tracing = use_tracing;
#endif
}
#endif

      This code was copied from my https://github.com/pythoncapi/pythoncapi_compat project. (I wrote the greenlet change.)

    27. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    28. iritkatriel commented on Sep 13, 2022

      @iritkatriel
      iritkatriel
      on Sep 13, 2022
      Member

      Is there anything left to do here?

    29. iritkatriel
      added
      pendingThe issue will be closed if no feedback is provided
      on Sep 13, 2022

    30. vstinner commented on Sep 15, 2022

      @vstinner
      vstinner
      on Sep 15, 2022
      Member

      IMO we are done, and I close the issue. See the issue #84128 for making PyThreadState opaque.

      The DISPATCH() macro is not as efficient as it could be

      This part was fixed early. Following comments were more about PyThreadState incompatible changes, how to migrate existing C extensions to Python 3.11 and how to design a new API which no longer access directly PyThreadState changes. In fact, that's already the topic of the issue #84128 that I created in 2020 and so we can continue the discussion there.

      The main change related to PyThreadState was the use_tracing member which was moved. I added PyThreadState_EnterTracing() and PyThreadState_LeaveTracing() functions to Python 3.11 for that and so projects already use it.

      I created #29121 to add PyThreadState_SetProfile() and PyThreadState_SetTrace() functions.

      I abandoned this PR. @pablogsal added PyEval_SetProfileAllThreads() and PyEval_SetTraceAllThreads() functions to Python 3.12 (commit e34c82a) which should fit the use case, with a different design.

    31. vstinner
      closed this as completedon Sep 15, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)pendingThe issue will be closed if no feedback is providedperformancePerformance or resource usagetopic-C-API

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions