Missing task done call in AsyncBulkCall

[giffels]

It seems that AsyncBulkCall is missing one task_done call. The simplify the logic, we can assume that size is set to 1.

In that case the first task is fetched in

tardis/tardis/utilities/asyncbulkcall.py

Line 134 in 75b60e3

results = [await queue.get()]

the while loop in

tardis/tardis/utilities/asyncbulkcall.py

Line 136 in 75b60e3

while len(results) < max_items and time.monotonic() < deadline:

is skipped.

and therefore task_done in

tardis/tardis/utilities/asyncbulkcall.py

Line 148 in 75b60e3

queue.task_done()

in never called.

More generally speaking there is always one task_done call missing independent of size.

[maxfischer2781]

Makes sense. We only have a queue.task_done() for each get in the loop, but none for the get outside it.

I'm guessing this isn't an immediate problem since nothing actually checks whether the queue is done. Still it would be useful to fix if we ever do anything that relies on this, e.g. shutting down the worker if there is no work at all.

[maxfischer2781]

Looks like there is also an issue that the bulk task could get garbage collected depending on its payload. We need to keep a strong reference to all tasks.