Repository navigation
Custom str subclass as keyword argument leads to System Error #94938
Description
When arguments are given by keyword using a dictionary, CPython requires that the key be of type str, or a sub-class. If a custom str subclass defines a badly-behaved __eq__, and is used that way, a SystemError (null argument) can result, which is not expected in the interpreter.
This report stems from a discussion on discuss.python.org about the types of keyword that ought to be allowed -- it's not code I actually want to run.
Reproducer
# kwargsbomb.py Use of a str subclass as a keyword
class S(str):
def __eq__(self, other):
print(other)
return True
def __hash__(self):
return 42
u = "my.domain.name"
a = u.split(**{'sep':'.'}) # ok
print(a)
b = u.split(**{S('sep'):'.'}) # ok
print(b)
c = u.split(**{S('xxx'):'.'}) # fails
print(c)
Output
['my', 'domain', 'name']
['my', 'domain', 'name']
sep
Traceback (most recent call last):
File " ... \kwargsbomb.py", line 18, in <module>
c = u.split(**{S('xxx'):'.'}) # fails
^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: null argument to internal routine
Environment
- CPython versions: 3.10.2 3.11.0b4 (in Idle and at the prompt)
- Operating system and architecture: Windows 10 64-bit
Activity
JelleZijlstra commentedon Jul 17, 2022
I'm not sure this should be supported. The discuss.python.org discussion was focused on strings that aren't valid identifiers, not on subclasses of
str.Restricting kwargs to only str may allow some optimizations and saves us from possible subtle bugs (what if a str subclass overrides
__hash__). In addition, there's no known use case for using str subclasses.I suggest we:
strserhiy-storchaka commentedon Jul 18, 2022
The user should not be able to provoke a SystemError from pure Python code. This exception is reserved for misuse of the C API. There is a bug somewhere in
_PyArg_UnpackKeywords(). In other examples it can lead to crash, memory corruption, or just silent produce incorrect result.jeff5 commentedon Jul 19, 2022
It starts with the question why integer keys are not allowed. When I looked where this is enforced, I was interested to find that it accepted sub-classes of
str. So I tried it, idly, and idle restarted (in 3.8).I agree, and favour that, but I cannot say for certain no-one is relying on it. Also @serhiy-storchaka is right that something is wrong elsewhere, and may go wrong in other circumstances, which this daft trick just serves to discover.
pythongh-94938: Fix errror detection of unexpected keyword arguments
serhiy-storchaka commentedon Jul 19, 2022
Thank you for your report @jeff5. It was an interesting bug.
gh-94938: Fix errror detection of unexpected keyword arguments (GH-94999
[3.11] pythongh-94938: Fix errror detection of unexpected keyword arg…
[3.10] pythongh-94938: Fix errror detection of unexpected keyword arg…
[3.11] gh-94938: Fix errror detection of unexpected keyword arguments (…
[3.10] gh-94938: Fix errror detection of unexpected keyword arguments (…
pythongh-94938: Fix test
gh-94938: Fix test (GH-95396)
pythongh-94938: Fix test (pythonGH-95396)
pythongh-94938: Fix test (pythonGH-95396)
gh-94938: Fix test (GH-95396)
gh-94938: Fix test (GH-95396)
merwok commentedon Aug 29, 2022
See also #96397