Repository navigation
3.11.5 Regression: Flags with custom __new__ can't be iterated #108682
Description
Bug report
Checklist
- I am confident this is a bug in CPython, not a bug in a third-party project
- I have searched the CPython issue tracker,
and am confident this bug has not been reported before
CPython versions tested on:
3.11
Operating systems tested on:
Linux
Output from running 'python -VV' on the command line:
Python 3.11.5 (main, Aug 25 2023, 23:47:33) [GCC 12.2.0]
A clear and concise description of the bug:
A behavior of Flag changed in python 3.11.5. When Flag with custom __new__ is used in combination with functional API, it's members can't be retrieved by iteration over the class.
MWE:
from enum import IntFlag
Perm = IntFlag('Perm', {'R': 4, 'W': 2, 'X': 1})
print(tuple(Perm))
class LabeledFlag(IntFlag):
def __new__(cls, value: int, label: str):
obj = super().__new__(cls, value)
obj._value_ = value
obj.label = label
return obj
LabeledPerm = LabeledFlag('LabeledPerm', {'R': (4, 'Read'), 'W': (2, 'Write'), 'X': (1, 'Exec')})
print(tuple(LabeledPerm))The output in python 3.11.4:
(<Perm.R: 4>, <Perm.W: 2>, <Perm.X: 1>)
(<LabeledPerm.R: 4>, <LabeledPerm.W: 2>, <LabeledPerm.X: 1>)
The output in python 3.11.5:
(<Perm.R: 4>, <Perm.W: 2>, <Perm.X: 1>)
()
I suspect this commit to introduce the regression: 59f009e
A workaround for 3.11.5:
class LabeledFlag(IntFlag):
def __new__(cls, value: int, label: str):
obj = super().__new__(cls, value)
obj._value_ = value
obj.label = label
return obj
@classmethod
def _missing_(cls, value):
pseudo_member = super(_DnskeyFlagBase, cls)._missing_(value)
if value in cls._value2member_map_ and pseudo_member.name is None:
cls._value2member_map_.pop(value)
return pseudo_memberLinked PRs
Metadata
Metadata
Assignees
Labels
Projects
- Status
Activity
ethanfurman commentedon Aug 31, 2023
This ended up being a combination of user error, program error, and some not-detailed-enough documentation.
An
enum's custom__new__should not usesuper(...).__new__(), because the parent's__new__is only for lookups; either use the data type directly:or, if using an already defined enum type (i.e.
LabeledFlag(IntFlag)and notLabeledFlag(int, Flag)):The code has been updated to raise a
TypeErrorifsuper().__new__is called, and a warning added to the docs.ethanfurman commentedon Aug 31, 2023
@ziima: the correct fix for your code:
which works across all Python versions.
ziima commentedon Aug 31, 2023
Works like a charm, thanks @ethanfurman.
I think I actually tried that, but I may had had other error in there at that moment.
gh-108682: [Enum] raise TypeError if super().__new__ called in custom…
pythongh-108682: [Enum] raise TypeError if super().__new__ called in …
[3.12] gh-108682: [Enum] raise TypeError if super().__new__ called in…
pythongh-108682: [Enum] raise TypeError if super().__new__ called in …
cdce8p commentedon Sep 6, 2023
Seems to be a breaking change. Not something would have expected in an
.rc2release to be honest.In particular raising a
TypeErrorfor enums without members worked fine before.Haven't had the time to look at it more closely, but noticed our test suite failing once again after it had been fixed for
3.12.0rc1, especially regarding the enum changes.[3.11] gh-108682: [Enum] raise TypeError if super().__new__ called in…
ethanfurman commentedon Sep 8, 2023
For posterity: the issue is that zigpy has empty enums, but uses a custom
_missing_to create their members on demand. This PR broke that, but I consider that behavior to be anti-enum, and keeping it also means keeping the bug in this issue.The solution for zigpy is to move the desired behavior into a custom metaclass.