Repository navigation
PyErr_SetObject() behavior is strange and not as documented. #101578
Description
Activity
The only case I can find of strange behavior leaking out is this:
import sys def wrap_in_sys_exit(ex): try: try: raise ex(-1) except ex as err: sys.exit(err) except BaseException as err: return err print(repr(wrap_in_sys_exit(ValueError))) print(repr(wrap_in_sys_exit(SystemExit)))
Which prints
SystemExit(ValueError(-1)) SystemExit(-1)instead of the expected
SystemExit(ValueError(-1)) SystemExit(SystemExit(-1))This API goes by to 1997. It is remarkable how simple and clear the code used to be:
void PyErr_Restore(PyObject *type, PyObject *value, PyObject *traceback) { PyThreadState *tstate = PyThreadState_GET(); PyObject *oldtype, *oldvalue, *oldtraceback; if (traceback != NULL && !PyTraceBack_Check(traceback)) { /* XXX Should never happen -- fatal error instead? */ Py_DECREF(traceback); traceback = NULL; } /* Save these in locals to safeguard against recursive invocation through Py_XDECREF */ oldtype = tstate->curexc_type; oldvalue = tstate->curexc_value; oldtraceback = tstate->curexc_traceback; tstate->curexc_type = type; tstate->curexc_value = value; tstate->curexc_traceback = traceback; Py_XDECREF(oldtype); Py_XDECREF(oldvalue); Py_XDECREF(oldtraceback); } void PyErr_SetObject(PyObject *exception, PyObject *value) { Py_XINCREF(exception); Py_XINCREF(value); PyErr_Restore(exception, value, (PyObject *)NULL); }
Reacted by Giampaolo RodolaThe code may have been simple, but it can leave the VM in a precarious state if the value is not a legal argument for calling the type.
- added a commit that references this issue
on Feb 9, 2023 The new C APIs
PyErr_GetRaisedExceptionandPyErr_SetRaisedExceptionare IMO problematic:- the former returns a borrowed reference
- the latter steals a reference
This goes against our recommendations for adding new public C APIs.
I also find the added documentation inadequate. It is not documented that the former function returns a borrowed reference. The documentation for the latter function is not exactly friendly written, and I'm specifically thinking of the last parenthesised sentence:
excmust be a valid exception. (Violating this rules will cause subtle problems later.) This call consumes a reference to the exc object: you must own a reference to that object before the call and after the call you no longer own that reference. (If you don’t understand this, don’t use this function. I warned you.)Suggesting to considering reopening this issue until at least the docs have been improved. IMO, we should also fix the newly added C APIs to be consistent in their reference handling, which means:
PyErr_GetRaisedExceptionshould return a new referencePyErr_SetRaisedExceptionshould not steal a reference
44 remaining items
- added a commit that references this issue
on Dec 31, 2023 - added a commit that references this issue
on Dec 31, 2023 I think this is all done. Thanks @iritkatriel and @erlend-aasland
- added a commit that references this issue
on Sep 23, 2024 - added a commit that references this issue
on Mar 18, 2026
Briefly:
PyErr_SetObject(exc_type, exc_val)does not create a new exception iffisinstance(exc_val, BaseException), but usesexc_valinstead.Callers of
PyErr_SetObject()need various workarounds to handle this.The long version:
Internally CPython handles exceptions as a triple
(type, value, traceback), but the language treats exceptions as a single value.This a legacy of the olden days before proper exceptions.
To handle adding proper exceptions to Python, various error handling functions, specifically
_PyErr_SetObjectstill treat exceptions as triples, with the convention that if the value is an exception, then the exception is already normalized.One other oddity is that if
exc_valis a tuple, it is treated as the*arguments toexc_typewhen calling it. So, ifisinstance(exc_val, BaseException)the desired behavior can be achieved by wrappingexc_valin a one-tuple.As a consequence, both
_PyErr_SetKeyErrorand_PyGen_SetStopIterationValueare a lot more complex than they should be to workaround this behavior.We could make
PyErr_SetObjectact as documented, but that is likely to break C extensions, given how old this behavior is, and that it is relied on throughout CPython.Code that does the following is common:
We could just document the current behavior, but the current behavior is strange.
What I suggest is this:
PyErr_SetObject()accuratelyThis is an old bug going back to the 2 series.
Linked PRs
Also relevant: