Repository navigation
Pluggable optimizer API #104584
Description
We need an API for optimizers to be plugged in to CPython.
The proposed model is that of client server, where the VM is the client and the optimizer is the server.
The optimizer registers with the VM, then VM calls the optimizer when hotspots are detected.
The API:
type struct {
OBJECT_HEADER;
_PyInterpreterFrame *(*execute)(PyExecutorObject *self, _PyInterpreterFrame *frame, PyObject **stack_pointer);
/* Data needed by the executor goes here, but is opaque to the VM */
} PyExecutorObject;
/* This would be nicer as an enum, but C doesn't define the size of enums */
#define PY_OPTIMIZE_FUNCTION_ENTRY 1
#define PY_OPTIMIZE_RESUME_AFTER_YIELD 2
#define PY_OPTIMIZE_BACK_EDGE 4
typedef uint32_t PyOptimizerCapabilities;
type struct {
OBJECT_HEADER;
PyExecutorObject *(*compile)(PyOptimizerObject* self, PyCodeObject *code, int offset);
PyOptimizerCapabilities capabilities;
float optimization_cost;
float run_cost;
/* Data needed by the compiler goes here, but is opaque to the VM */
} PyOptimizerObject;
void _Py_Executor_Replace(PyCodeObject *code, int offset, PyExecutorObject *executor);
int _Py_Optimizer_Register(PyOptimizerObject* optimizer);The semantics of a PyExecutorObject is that upon return from its execute function, the VM state will have advanced N instructions. Where N is a non-negative integer.
Full discussion here: faster-cpython/ideas#380
This is not a replacement for PEP 523. That will need a PEP. We should get this working first, before we consider replacing PEP 523.
Linked PRs
- GH-104584: Optimizer API #105100
- GH-104584: Allow optimizers to opt out of optimizing. #105244
- GH-104584: Assorted fixes for the optimizer API. #105683
- gh-104584: Baby steps towards generating and executing traces #105924
- gh-104584: Add #line directives to executor_cases.c.h #106126
- gh-104584: Fix assert in DEOPT macro -- should fix buildbot #106131
- GH-104584: Fix
ENTER_EXECUTOR#106141 - gh-104584: Change DEOPT_IF in uops executor #106146
- gh-104584: Emit macro expansions to opcode_metadata.h #106163
- GH-104584: Fix test_capi.test_counter_optimizer() when run twice #106171
- GH-104584: Add Include/cpython/optimizer.h to Makefile.pre.in #106277
- gh-104584: Support most jumping instructions #106393
- gh-104584: Fix error handling from backedge optimization #106484
- gh-104584: Handle EXTENDED_ARG in superblock creation #106489
- gh-104584: Clean up and fix uops tests and fix crash #106492
- gh-104584: Allow unspecialized instructions in superblocks #106497
- gh-104584: Move super-instruction special-casing to generator #106500
- gh-104584: Replace ENTER_EXECUTOR with the original in trace projection #106526
- gh-104584: readability improvements in optimizer.c #106641
- GH-104584: Miscellaneous fixes for
-Xuops#106908 - GH-104584: Fix incorrect uoperands #107513
- GH-104584: Restore
frame->stacktopon optimizer error #108953 - GH-104584: Remove
ip_offsetand simplifyEXIT_TRACE#108961 - GH-104584: Don't call executors from
JUMP_BACKWARD#109347 - GH-104584: Fix refleak when tracing through calls #110593
Activity
markshannon commentedon May 18, 2023
Note that the above API is just the initial version to support our work on speeding up Python 3.13.
It will probably need to be extended to support PyTorch Dynamo and other users of PEP 523 that cannot use PEP 669, but that is for another issue.
GH-104584: Plugin optimizer API (GH-105100)
GH-104584: Allow optimizers to opt out of optimizing. (GH-105244)
GH-104584: Assorted fixes for the optimizer API. (GH-105683)
gh-104584: Baby steps towards generating and executing traces (#105924)
gh-104584: Add #line directives to executor_cases.c.h (#106126)
gh-104584: Fix assert in DEOPT macro -- should fix buildbot (#106131)
gh-104584: Change DEOPT_IF in uops executor (#106146)
pythonGH-104584: Fix test_capi.test_counter_optimizer() when run twice
GH-104584: Fix test_capi.test_counter_optimizer() when run twice (#10…
gh-104584: Emit macro expansions to opcode_metadata.h (#106163)
pythonGH-104584: Add Include/cpython/optimizer.h to Makefile.pre.in
GH-104584: Add Include/cpython/optimizer.h to Makefile.pre.in (#106277)
9 remaining items
GH-104584: Miscellaneous fixes for -Xuops (GH-106908)
GH-104584: Fix incorrect uoperands (GH-107513)
GH-104584: Restore frame->stacktop on optimizer error (GH-108953)
GH-104584: Don't call executors from JUMP_BACKWARD (GH-109347)
GH-104584: Fix refleak when tracing through calls (GH-110593)
encukou commentedon Jun 20, 2024
Is documenting these planned for 3.13?
markshannon commentedon Jun 20, 2024
Since we are removing them in 3.14, probably not.
vstinner commentedon Jun 20, 2024
If possible, I would like to backport #120643 to Python 3.13, to solve a C99 compatibility issue.
pythonGH-104584: Fix refleak when tracing through calls (pythonGH-110593
erlend-aasland commentedon Oct 13, 2024
A bunch of PRs were merged; a quick glance at the PR list suggests that this issue can be closed as completed. Are there further work to be done?
brandtbucher commentedon Nov 7, 2024
We discussed the optimizer API offline. In short: we should rip the API itself out, and just keep all of the code that does that actual optimizing.
The API itself is constantly changing, poorly-defined, and undocumented. There's a bunch of infrastructure required just to test the API (not the actual optimizations we perform), and it introduces indirection and artificial boundaries into some pretty performance-sensitive stuff. Nobody it using it that we're aware of, nobody we've talked to is planning on using it, and frankly we don't want anyone to start using it. So let's remove it.
vstinner commentedon Nov 8, 2024
Nitpick: I suggest opening a new issue to remove it.
brandtbucher commentedon Nov 8, 2024
GH-126599