non-awaitable object makes asyncio.wait hang if there are async subprocess tasks
#105288
Comments
|
Why are you passing a non awaitable object in the first place? |
As you can see, that's a misuse. It reveals two problems:
|
|
The issue here isn't because of import asyncio
from asyncio import subprocess
async def whoami():
print('Finding out who am I...')
proc = await subprocess.create_subprocess_exec(
'whoami',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
print('Waiting for the process to finish...', proc)
stdout, _ = await proc.communicate()
print(f'I am {stdout}')
async def main():
t2 = asyncio.create_task(whoami())
await asyncio.sleep(0)
# await asyncio.sleep(0) Uncomment to fix hang
if __name__ == '__main__':
asyncio.run(main())As seen in the code adding a sleep(0) which is essentially cycle event loop once before tearing it apart fixes it. |
Thanks for your detailed explanation. But I think we are a little off-topic. Do you mean that I should "fix" the litmus program rather than fix Let go back to the track by printing the lifetime of the litmus program and compare before/after the commit Let's a line of code ntodo = len(self._ready)
for i in range(ntodo):
handle = self._ready.popleft()
print('Handle:', handle, file=sys.stderr) # <---- add a log here
if handle._cancelled:
continueAnd add another line # wake up futures waiting for wait()
print('Waking up exit waiters...', file=sys.stderr) # <---- add a log here
for waiter in self._exit_waiters:
if not waiter.cancelled():
waiter.set_result(returncode)
self._exit_waiters = NoneTo synchronize all outputs, we also add Current behaviorWhen running without the PR, it prints: It's clear that the main program hangs, while the subprocess has exited. The last "good" revision, before
|
I have reopened the PR (I meant to request changes but somehow it got closed and I didn't look again sorry for that ) though I wasn't referring to the test failure added by your test but rather other tests. I referred to the fix as wrong as it basically reverts my fix. That change isn't straight forward to understand, see my explanation of it in #88050 (comment) |
|
For your reference, the test failure I was referring to were from https://github.com/python/cpython/actions/runs/5212830335/jobs/9406983274, see the warnings in other tests not added by your PR. |
|
This problem can be avoided by using a TaskGroup: import asyncio
from asyncio import subprocess
async def whoami():
print('Finding out who am I...')
proc = await subprocess.create_subprocess_exec(
'whoami',
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
)
stdout, _ = await proc.communicate()
print(f'I am {stdout}')
async def main():
async with asyncio.TaskGroup() as tg:
tg.create_task(asyncio.sleep(0))
t2 = tg.create_task(whoami())
# Both `[0, t1]` and `[0, t2]` can cause the problem
await asyncio.wait([0, t2])
if __name__ == '__main__':
asyncio.run(main())this is a duplicate of #103847 |
asyncio.waitaccepts aIterable[Awaitable[_T]]as its first parameter. However it will hang forever if we passes an invalidAwaitableto it and there are subprocesses wrapped inasyncio.Taskrunning.To sum up, we have to meet these conditions at once:
asyncio.subprocess. It does not matter if weasyncio.waitfor it or not.asyncio.waiton iterators that can yield non-awaitable objects (such as integers).Here is a litmus program that reproduces this problem:
When running it on Linux, this program is expected to terminate immediately after throwing an exception. Because we are attempting to wait for
0, which does not implementAwaitable[_T]. However the program will barely output:and hangs forever. Hitting
Ctrl-Cdoes terminate it, but the stack trace points to somewhere far away from the real cause:It does not matter if we wait for
[0, t1], or[0, t2]. As long ast2(task of the subprocess) is created, the program will hang when awaiting on that list.If we run it on previous versions of Python (such as the latest branch
3.10,2c9b0f30), it will cause an exception thrown in the standard library, though the exact behavior seems to be undefined, since I have observed many different locations that throws some kind of exceptions likeAttributeErrorin different locations, when bisecting using git. The correct output looks like:My test environment is Intel 12th gen core with the latest Archlinux & zen kernel (
6.3.5-zen1-1-zen). I have rerun the test on an updated Debian 11 VM and got the same result. So I don't think it is related to distros and kernel versions.Bisecting shows this issue was directly introduced in commit
7015e137:gh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (#32073), which attempted to fix a previousasyncioissue. After investigating the source code, I believe this was caused by this commit, which has two major problems:pipe. If we look at method_try_finish, we find that method has validated the same objects, so this should be fixed. This causes an unrelated exception and interrupts the clean-up process_process_exited. However this piece of code has been removed in a subsequent commit. So it is not relevant today.self._exit_waitersafter subprocess exited. This directly causes the program hang forever.I will post my fix and detailed explanation in a PR shortly. Please let me know if there are problems.
Linked PRs
The text was updated successfully, but these errors were encountered: