Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List comprehensions sometimes call iter(iter(v)) instead of just iter(v) #128211

Closed
DemiMarie opened this issue Dec 24, 2024 · 3 comments
Closed
Labels
type-bug An unexpected behavior, bug, or error

Comments

@DemiMarie
Copy link

DemiMarie commented Dec 24, 2024

Bug report

Bug description:

A development version of the Qubes OS updater gave the following traceback:

Dec 23 21:53:01 dom0 widget-wrapper[25404]: Traceback (most recent call last):
Dec 23 21:53:01 dom0 widget-wrapper[25404]:   File "/usr/lib/python3.13/site-packages/qui/updater/updater.py", line 215, in next_clicked
Dec 23 21:53:01 dom0 widget-wrapper[25404]:     vms_to_update = self.intro_page.get_vms_to_update()
Dec 23 21:53:01 dom0 widget-wrapper[25404]:   File "/usr/lib/python3.13/site-packages/qui/updater/intro_page.py", line 147, in get_vms_to_update
Dec 23 21:53:01 dom0 widget-wrapper[25404]:     return self.list_store.get_selected()
Dec 23 21:53:01 dom0 widget-wrapper[25404]:            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
Dec 23 21:53:01 dom0 widget-wrapper[25404]:   File "/usr/lib/python3.13/site-packages/qui/updater/utils.py", line 306, in get_selected
Dec 23 21:53:01 dom0 widget-wrapper[25404]:     selected_rows = [row for row in self if row.selected]
Dec 23 21:53:01 dom0 widget-wrapper[25404]:                                     ^^^^
Dec 23 21:53:01 dom0 widget-wrapper[25404]: TypeError: 'UpdateListIter' object is not iterable

However, this makes no sense. The code can be found at https://github.com/QubesOS/qubes-desktop-linux-manager/blob/a3e4d177785bbc1b1ee72bd7cd83ef7b8a4e8827/qui/updater/utils.py#L300-L309, where self is an ListWrapper instance. Its __iter__ method returns an UpdateListIter, which is a valid iterator. However, UpdateListIter does not have its own __iter__ method. Adding a method that returns self makes the problem go away.

A small reproducer is below:

class B:
    def __next__(self):
        raise StopIteration
class A:
    def __iter__(self):
         return B()
# works
for i in A():
    print(i)
# BUG: should print [], but instead raises TypeError
print([i for i in A()])

On Python 3.12, this works (and prints []). On Python 3.13, this raises TypeError.

At least some standard library iterators (such as iter([]) have an __iter__(self) that returns self, as shown by the following program printing True on both 3.13 and 3.14:

a = iter([])
print(a is iter(a))

CPython versions tested on:

3.13

Operating systems tested on:

Linux

@DemiMarie DemiMarie added the type-bug An unexpected behavior, bug, or error label Dec 24, 2024
DemiMarie added a commit to DemiMarie/qubes-desktop-linux-manager that referenced this issue Dec 24, 2024
Python 3.13 list comprehensions are buggy [1]: [i for i in x] calls
iter(iter(x)) instead of just iter(x).  Work around the bug by having
UpdateListIter have a __iter__(self) that just returns self.  This is
the same thing that the standard library does:

    x = iter([])
    print(x is iter(x))

prints True.

[1]: python/cpython#128211
@TeamSpen210
Copy link

See #89413 and #29170 - in 3.11, the docs were changed to indicate that iterators must define __iter__ also, but the interpreter hasn't reliably checked for this. This was also brought up recently in #128161. 3.13 is just taking advantage of this to fix a different issue.

@Eclips4
Copy link
Member

Eclips4 commented Dec 24, 2024

Duplicate of #128161

@Eclips4 Eclips4 marked this as a duplicate of #128161 Dec 24, 2024
@Eclips4 Eclips4 closed this as not planned Won't fix, can't repro, duplicate, stale Dec 24, 2024
@terryjreedy
Copy link
Member

The idea that iterators should be iterable is not recent, but was part of the design introduced, I believe, in 2.2.

DemiMarie added a commit to DemiMarie/qubes-desktop-linux-manager that referenced this issue Dec 31, 2024
Python iterators are required to implement an __iter__() that returns
self [1].  CPython doesn't check this consistently, but the requirement
is still there, so the existing code is buggy.  Python 3.13 started
checking this in list comprehensions, resulting in exceptions being
thrown.  Fix the bug by having __iter__() return self, as required by
the iterator protocol.

This worked on Python 3.13 and below, but broke in 3.13.1 [2].

[1]: https://docs.python.org/3/glossary.html#term-iterator
[2]: python/cpython#128211
DemiMarie added a commit to DemiMarie/qubes-desktop-linux-manager that referenced this issue Jan 4, 2025
Python iterators are required to implement an __iter__() that returns
self [1].  CPython doesn't check this consistently, but the requirement
is still there, so the existing code is buggy.  Python 3.13 started
checking this in list comprehensions, resulting in exceptions being
thrown.  Fix the bug by having __iter__() return self, as required by
the iterator protocol.

This worked on Python 3.13 and below, but broke in 3.13.1 [2].

[1]: https://docs.python.org/3/glossary.html#term-iterator
[2]: python/cpython#128211
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants