Part of POST data is lost under hightload

[POD666]

Short story

I was trying to extract POST data from request object passed to new task.
Everything is ok for most of requests but sometimes part of data is missing.
I hope code will explain better.

Steps to reproduce

async def fire_and_forget(request):
    data_raw = await request.text()
    if data_raw:
        try:
            data = json.loads(data_raw)
        except json.JSONDecodeError:
            log.error('JSONDecodeError: <|%s|>' % data_raw)   # log what was extracted from request
    else:
        data = {}
   #...

async def handle_post(request):
    feature = asyncio.ensure_future(fire_and_forget(request), loop=asyncio.get_event_loop())
    return web.Response(text="")

Bash test:

$ seq 100000 | parallel -j30 --no-notice "curl -X POST -H 'Content-Type: application/json' -d @post_data.json -s -w %{time_total} http://localhost/myurl/?{}"

Here is content of post_data.json: https://pastebin.com/raw/PKRFVqKC
Here is what I get sometimes: https://pastebin.com/raw/s2Mq6EeN

Your environment

I'm running docker-compose, cloned from https://github.com/Zeliboba5/aiohttp-boilerplate
$pip freeze:
aenum==2.0.8
aiohttp==3.0.1
async-timeout==2.0.1
asyncpg==0.13.0
attrs==17.4.0
chardet==3.0.4
idna==2.6
idna-ssl==1.0.1
marshmallow==2.13.5
multidict==4.1.0
PyPika==0.10.1
PyYAML==3.12
uvloop==0.9.1
yarl==1.1.1

[socketpair]

You can not receive request data after you instruct aiohttp to send response. You code is wrong, just read data using await in request handler . This will work "in background" actually

[asvetlov]

Looks like buggy github mobile did the right thing: the code example is not valid, the issue should be closed

[asvetlov]

Looks like buggy github mobile did the right thing: the code example is not valid, the issue should be closed

[POD666]

I have fixed this like you suggested. Thanks.
But I think warning or exception is needed when trying to receive request data after response was sent instead of returning half of POST data.

[asvetlov]

I like the idea. Created #2895 for this

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new