Enum regression: AttributeError when accessing class variables on instances

[hroncok]

BPO	43162
Nosy	@ethanfurman, @hroncok
PRs	bpo-43162: [Enum] deprecate enum member.member access #24486 bpo-43162: [Enum] update docs, renable doc tests #24487

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ethanfurman'
closed_at = <Date 2021-03-03.19:42:57.050>
created_at = <Date 2021-02-08.12:22:59.718>
labels = ['type-bug', 'library', '3.10']
title = 'Enum regression: AttributeError when accessing class variables on instances'
updated_at = <Date 2021-03-03.22:22:26.284>
user = 'https://github.com/hroncok'

bugs.python.org fields:

activity = <Date 2021-03-03.22:22:26.284>
actor = 'ethan.furman'
assignee = 'ethan.furman'
closed = True
closed_date = <Date 2021-03-03.19:42:57.050>
closer = 'ethan.furman'
components = ['Library (Lib)']
creation = <Date 2021-02-08.12:22:59.718>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 43162
keywords = ['patch']
message_count = 13.0
messages = ['386626', '386628', '386642', '386668', '386672', '386674', '386680', '386683', '386700', '387953', '388038', '388053', '388063']
nosy_count = 3.0
nosy_names = ['ethan.furman', 'hroncok', 'baker.dylan.c']
pr_nums = ['24486', '24487']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43162'
versions = ['Python 3.10']

[hroncok]

I believe I found a regression in Enum in Python 3.10.0a5.

This is Python 3.9:

>>> import enum
>>> class C(enum.Enum):
...     A = 0
...     B = 1
... 
>>> C.A
<C.A: 0>
>>> C.B
<C.B: 1>
>>> C(0).A
<C.A: 0>
>>> C(0).B
<C.B: 1>
>>> 

The Enum instances can access class-attributes via dot, like normal instances do.

While in Python 3.10.0a5:

>>> import enum
>>> class C(enum.Enum):
...     A = 0
...     B = 1
... 
>>> C.A
<C.A: 0>
>>> C.B
<C.B: 1>
>>> C(0).A
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.10/enum.py", line 146, in __get__
    raise AttributeError(
AttributeError: C: no attribute 'A'
>>> C(0).B
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python3.10/enum.py", line 146, in __get__
    raise AttributeError(
AttributeError: C: no attribute 'B'

In real word situations, it breaks meson:

https://github.com/mesonbuild/meson/blob/398df5629863e913fa603cbf02c525a9f501f8a8/mesonbuild/backend/backends.py#L52-L78

The __str__ method does:

    if self is self.EXITCODE: ...

And it fails with:

AttributeError: TestProtocol: no attribute 'EXITCODE'

This worked with 3.10.0a4.

If this is a deliberate backwards incompatible change of behavior, I don't think it is documented in the changelog or what's new in Python 3.10, nor that it was deprecated in Python 3.9 and 3.8.

[hroncok]

Git bisect:

c314e60 is the first new commit
commit c314e60
Author: Ethan Furman <[email protected]>
Date: Tue Jan 12 23:47:57 2021 -0800

bpo-42901: [Enum] move member creation to `__set_name__` (GH-24196)

`type.__new__` calls `__set_name__` and `__init_subclass__`, which means
that any work metaclasses do after calling `super().__new__()` will not
be available to those two methods.  In particular, `Enum` classes that
want to make use of `__init_subclass__` will not see any members.

Almost all customization is therefore moved to before the
`type.__new__()` call, including changing all members to a proto member
descriptor with a `__set_name__` that will do the final conversion of a
member to be an instance of the `Enum` class.

Lib/enum.py | 297 +++++++++++++--------
Lib/inspect.py | 2 +-
Lib/test/test_enum.py | 15 +-
.../2021-01-11-17-36-59.bpo-42901.gFd-ta.rst | 3 +
4 files changed, 207 insertions(+), 110 deletions(-)
create mode 100644 Misc/NEWS.d/next/Library/2021-01-11-17-36-59.bpo-42901.gFd-ta.rst

[ethanfurman]

The code for that __str__ seems very inefficient -- why doesn't it just do:

    return self.name

?

-----

The issue is not being able to access class attributes, the issue is whether one enum member should be seen as an attribute of another:

EnumClass.RED.BLUE

and the answer is no. That wasn't possible when Enum was first introduced in 3.4, and was an unfortunate side-effect of speeding up member access in 3.5 (or 3.6). The docs have always warned against it.

A deprecation warning is an easier transition, though, so I'll do that for 3.10, and in 3.11 it will become an error.

Thank you for reporting! :-)

[hroncok]

Thanks. In the meantime, I've opened mesonbuild/meson#8318

[bakerdylanc]

Author of said meson code here. I use this pattern when the enum names and values are implementation details, but the string values are user visible. In this case the enum itself is used internally to represent what kind of test we're doing, but we're initializing it from user input. There might be reasons in the future that the names of the enum members and the string values being passed in aren't the same anymore, say because we map two string values to one enum value. I guess I could accomplish the same thing with a staticmethod or helper function, but the code is effectively an alternate initializer, and follows that pattern using a classmethod.

[hroncok]

Ethan, should the depreciation exist for 2 releases prior to removal?

Dylan, even in that case, I guess the proper way to access the other members is supposed to be type(self).FOO or ClassName.FOO, not self.FOO.

[ethanfurman]

New changeset d65b903 by Ethan Furman in branch 'master':
bpo-43162: [Enum] deprecate enum member.member access (GH-24486)
d65b903

[ethanfurman]

Dylan, it's not the from_str() method, but the __str__ method that is the problem. Instead of

    def __str__(self):
        if self is self.EXITCODE: 
            return 'exitcode' 

it should be

    def __str__(self):

        cls = self.__class__

        if self is cls.EXITCODE: 
            return 'exitcode'

[hroncok]

"Wait for the warning to appear in at least two major Python versions. It's fine to wait more than two releases."

https://www.python.org/dev/peps/pep-0387/#basic-policy-for-backwards-compatibility

So indeed, if you add the warning in 3.10, the behavior can be removed from 3.12 earliest.

[ethanfurman]

DeprecationWarning will be active in 3.10 and 3.11 with removal in 3.12.

[ethanfurman]

New changeset 44e580f by Ethan Furman in branch 'master':
bpo-43162: [Enum] update docs, renable doc tests (GH-24487)
44e580f

[hroncok]

Thank you, Ethan.

[ethanfurman]

You're welcome. Thank you for pushing the issue! :-)