Skip to content

Trace recording JIT #139109

Open
Open
Trace recording JIT#139109
Assignees
Fidget-Spinner
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-JITtype-featureA feature request or enhancement

Description

@Fidget-Spinner
Fidget-Spinner
opened on Sep 18, 2025

This issue tracks progress of changing the JIT from one that projects traces to one that records them.

Follow-ups:

  • Embed ENTER_EXECUTOR's executor in the bytecode and on trace graphs?
  • Do not increase chain depth when side-exiting a branch, as that is just normal control-flow.
  • Optimize through CALL_ALLOC_AND_ENTER_INIT
  • Specialize: CALL_FUNCTION_EX, __init__ of slots, SEND for yield from
  • Loop peel.

Linked PRs

Activity

  1. Fidget-Spinner
    added
    type-featureA feature request or enhancement
    on Sep 18, 2025
  2. Fidget-Spinner
    self-assigned this
    on Sep 18, 2025
  3. Fidget-Spinner
    added
    topic-JIT
    on Sep 18, 2025
  4. Fidget-Spinner
    added a commit that references this issue on Sep 18, 2025

    gh-139109: Dynamic opcode targets (GH-139111)

    a269e69
  5. StanFromIreland
    added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    on Sep 18, 2025

  6. markshannon commented on Sep 23, 2025

    @markshannon
    markshannon
    on Sep 23, 2025
    Member

    Just to flesh this out a bit:

    Motivation

    Currently we project traces from the initial hotspot, but there are a number of places in the projection where we would benefit from having live values as well as the statistical information gathered by the interpreter.

    We can combine the information from live values with the statistical information to give us a trace that should, in theory, be better than from either pure trace recording or from projection.

    Design

    Confidence

    As a trace gets longer, it is less likely that a future execution will remain on trace. Our estimate of that future likelihood is our "confidence" in the trace at that point. All traces start with confidence at 100%, but that confidence drops with each guard in the trace.

    The trace recording interpreter

    The trace recording interpreter operates exactly as the normal tier 1 interpreter, while recording the instructions executed.

    While trace recording we should maintain a confidence level, but this confidence level will not be as accurate as the optimizer can provide. Since the optimizer might be more confident, and we cannot optimize beyond the end of the recorded trace, the confidence limit for recording should be lower than for the final trace.

    Possible implementations

    We can:

    1. Record the trace at the bytecode level, then lower to micro-ops as a separate pass
    2. Lower as we go. This is more complicated, but may be faster as it saves recording the bytecode.

    I think we should start with 1. We can move to 2. later without impacting the rest of the JIT.

    To avoid doubling the size of the interpreter, each recording opcode implementation should call a common recording function then jump to the normal implementation of that opcode.

    However, some specialized instructions could "deopt", executing the non-specialized instruction. This would result in a misleading trace, so we might want to somehow force re-specialization in that case. It is not obvious how to do that without duplicating all specialized instructions. We could force re-specialization of all instructions, that might be slow.

  7. markshannon commented on Oct 29, 2025

    @markshannon
    markshannon
    on Oct 29, 2025
    Member

    @Fidget-Spinner
    We can keep the switch interpreter working with the tracing JIT using the same technique as we did for sys.settrace tracing in 3.11.

    • Change Tools/cases_generator/analyzer.py to change ENTER_EXECUTOR to 254, and lower the number of all the instrumented instructions by one, leaving 255 as reserved. Call opcode 255 INTERNAL_TRACING.
    • In PyEval_EvalDefault add a local variable tracing which is set to 0 if tracing is off and 255 if tracing is on.
    • The INTERNAL_TRACING instruction should jump to record_previous_inst which should dispatch using the plain opcode without tracing. It will be unused for computed gotos and tailcalling.
    • Dispatch is a bit more complicated. Instead of dispatching on opcode, dispatch on dispatch_opcode. Before the goto dispatch, the various DISPATCH macros need to set dispatch_opcode:
      • Non-tracing dispatch: dispatch_opcode = opcode
      • Normal dispatch: dispatch_opcode = opcode | tracing
      • Tracing dispatch: dispatch_opcode = 255

    Removing the 3.11 scheme was worth about 1% speedup, so adding it back will slow the interpreter at least 1% as this is more complex than the 3.11 scheme. However, if we make the JIT faster, we should get that back and more.

  8. markshannon commented on Nov 11, 2025

    @markshannon
    markshannon
    on Nov 11, 2025
    Member

    Once #140310 is merged, there are a few more tasks to do:

    • Restore switch based interpreter
    • Record values, so we can get rid of the ad-hoc caches for functions, classes and code objects.
    • Document how it works
    • Improve efficiency and thread safety by storing the buffer on interpreter and swapping it to the thread when needed.
    • Restore some sort of confidence estimate to tracing or optimization or both, to avoid overly long traces
    • Restore the optimization for stack size checks and its tests
    • Remove some code duplication by setting the previous instruction to NOP when entering tracing.

  9. Fidget-Spinner commented on Nov 11, 2025

    @Fidget-Spinner
    Fidget-Spinner
    on Nov 11, 2025
    MemberAuthor

    The only test we need to restore is the stack size optimization checks. That's all the tests that are disabled for now.

  10. Fidget-Spinner commented on Nov 13, 2025

    @Fidget-Spinner
    Fidget-Spinner
    on Nov 13, 2025
    MemberAuthor

    Another action item: prime number table for the backoff counters. This will allow for a trace of a "better" iteration. Note that pypy also uses a "strange" number for warmup of loops.

  11. markshannon commented on Nov 13, 2025

    @markshannon
    markshannon
    on Nov 13, 2025
    Member

    #141498

  12. Fidget-Spinner
    added a commit that references this issue on Nov 13, 2025

    gh-139109: A new tracing JIT compiler frontend for CPython (GH-140310)

    4fa80ce
  13. CuriousLearner
    added 3 commits that reference this issue on Nov 13, 2025

    Merge branch 'main' into fix-issue21914

    b9f473c

    Merge branch 'main' of github.com:python/cpython into fix-Issue18387

    c46ecc5

    Merge branch 'main' of github.com:python/cpython into fix-issue24255

    c240289

  14. 7 remaining items

  15. Fidget-Spinner commented on Dec 10, 2025

    @Fidget-Spinner
    Fidget-Spinner
    on Dec 10, 2025
    MemberAuthor

    Yeah lets do follow ups separately

  16. Fidget-Spinner
    closed this as completedon Dec 10, 2025

  17. markshannon commented on Dec 16, 2025

    @markshannon
    markshannon
    on Dec 16, 2025
    Member

    I'd rather keep this open, otherwise it gives a false impression of being finished when there's some clean up to do.
    Or, make new issues for the list above and then close this one.

  18. markshannon
    reopened this on Dec 16, 2025

  19. Fidget-Spinner commented on Dec 16, 2025

    @Fidget-Spinner
    Fidget-Spinner
    on Dec 16, 2025
    MemberAuthor

    Restore the optimization for stack size checks and its tests

    I think we should abandon this and just clean up the code implementing this optimisation. It's generally very finnicky and I don't think it nets us much perf wins.

  20. Fidget-Spinner
    added a commit that references this issue on Dec 22, 2025

    gh-139109: Add terminator to JIT code when halting due to invalid dep…

    665d280
  21. markshannon
    added a commit that references this issue on Jan 22, 2026

    GH-139109: Partial reworking of JIT data structures (GH-144105)

    d77aaa7

  22. cocolato commented on Feb 2, 2026

    @cocolato
    cocolato
    on Feb 2, 2026
    Member

    I think we should abandon this and just clean up the code implementing this optimisation. It's generally very finnicky and I don't think it nets us much perf wins.

    Hi, @Fidget-Spinner
    Refer: #144388. Should we convert _CHECK_STACK_SPACE to _CHECK_STACK_SPACE_OPERAND one by one? If we need this, i'll implement it.

  23. markshannon
    added a commit that references this issue on Feb 2, 2026

    gh-139109: Replace `_CHECK_STACK_SPACE` with `_CHECK_STACK_SPACE_OPER…

    89e6607
  24. Aniketsy
    added a commit that references this issue on Feb 3, 2026

    pythongh-139109: Replace `_CHECK_STACK_SPACE` with `_CHECK_STACK_SPAC…

    9c07998
  25. thunder-coding
    added 2 commits that reference this issue on Feb 15, 2026

    pythonGH-139109: Partial reworking of JIT data structures (pythonGH-1…

    8bcb200

    pythongh-139109: Replace `_CHECK_STACK_SPACE` with `_CHECK_STACK_SPAC…

    9d90bcc
  26. Fidget-Spinner
    added a commit that references this issue on Mar 31, 2026

    gh-139109: Document the trace recording interpreter in internaldocs (G…

    362145c
  27. clin1234
    added a commit that references this issue on Apr 16, 2026

    pythongh-139109: Document the trace recording interpreter in internal…

    b530541
  28. ljfp
    added a commit that references this issue on Apr 25, 2026

    pythongh-139109: Document the trace recording interpreter in internal…

    3016cf1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-JITtype-featureA feature request or enhancement

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions