Odd error awaiting a Future

[dimaqq]

BPO	37172
Nosy	@asvetlov, @dimaqq, @1st1

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-06-06.10:29:59.344>
labels = ['3.7', '3.8', '3.9', 'expert-asyncio']
title = 'Odd error awaiting a Future'
updated_at = <Date 2019-06-10.10:18:40.184>
user = 'https://github.com/dimaqq'

bugs.python.org fields:

activity = <Date 2019-06-10.10:18:40.184>
actor = 'Dima.Tisnek'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2019-06-06.10:29:59.344>
creator = 'Dima.Tisnek'
dependencies = []
files = []
hgrepos = []
issue_num = 37172
keywords = []
message_count = 9.0
messages = ['344798', '344800', '345017', '345100', '345102', '345111', '345114', '345115', '345118']
nosy_count = 3.0
nosy_names = ['asvetlov', 'Dima.Tisnek', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue37172'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[dimaqq]

Let's start with correct code:

import asyncio


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    global g1
    g1 = asyncio.Future()
    await asyncio.gather(reader(), writer())

asyncio.run(test())

No error, as expected.

Now let's mess it up a bit:

import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), writer())

asyncio.run(test())

Fails with RuntimeError ... attached to a different loop

The error makes sense, although it's sad that I can't create global futures / there was no even loop when Future was creates, it was not a *different* event loop / maybe I wish .run() didn't force a new event loop?

A nit (IMO), but I can live with it.

Let's mess the code up a bit more:

import asyncio

g1 = asyncio.Future()


async def writer():
    await asyncio.sleep(1)
    g1.set_result(41)

async def reader():
    await g1

async def test():
    await asyncio.gather(reader(), reader(), writer())

asyncio.run(test())

RuntimeError: await wasn't used with future

What?
That's really confusing!
The only difference is that there are now 2 readers running in parallel.

The actual exception comes from asyncio.Future.__await__ after a yield.
I'm not sure how to fix this...

[asvetlov]

Global future objects (and global asyncio objects in general) don't work with asyncio.run(). The lifecycle of these objects should be closer than loop but asyncio.run() creates a loop during execution.

I think this is a good design, we have a plan to deprecate and eventually drop all old-times mess that allows confusions.

All three of your problems are because you use a global future which is implicitly attached to the different loop.

Also I'd like to note that futures are low-level API, which is very delicate and error-prone. Futures are crucial for libraries building (e.g. aiohttp uses them a lot) but working with futures in application code is an explicit sign of bad design.

[terryjreedy]

Dima, unless you want to make a specific doc change suggestion, I think this should be closed. The planned code changes will be on other issues.

[dimaqq]

Hi Terry,

Yes, I have a specific suggestion:

The error RuntimeError: await wasn't used with future is misleading.
I'm not sure if changing error text is enough.
I think that Future.await should be fixed; I think that await f should be idempotent, that the same exception should be raised in single and double await examples.

[terryjreedy]

OK. I am quitting here because asyncio and futures are outside my expertise.

[asvetlov]

Not sure what "idempotency fix" means in the context of Future objects.

Could you describe desired behavior or, even better, provide a pull request that demonstrates your desire?

[dimaqq]

I think that if a Future is abused, the following two should return *same* error:

async def test():
    await asyncio.gather(reader(), writer())

-vs-

async def test():
    await asyncio.gather(reader(), reader(), writer())

[asvetlov]

Agree, it would be fine.
So the behavior is not changed but the error text should be clear, right?

[dimaqq]

Yes, I think that would be sufficient!

I don't know about implementation though, as there's some magic in the way Future.__await__ tries to guess how it was called. I hope one day that magic could go away, but I neither understand the origin nor the mechanics of it, so... My gut tells me that fixing the error involves digging into that magic.

[kumaraditya303]

This is definitely one of the obscure edge cases with mixing global futures and different loop. I checked and this bug is only present in the C implementation so a bug in _asynciomodule.c.

[kumaraditya303]

Let's not spend more time on this, the easier way to avoid this is to just avoid using global futures which isn't recommended either.