Fixed possible starvation issue during upgrading to WebSocket transport

[ntamas]

This pull request fixes a possible starvation issue that prevents the socket from being upgraded from polling to WebSocket transport if the outbound packet queue is filled too fast from other (green) threads.

I have stumbled upon the issue while building a WebSocket service that generates packets with a high frequency. In our case, the packets represent measurements that come from external devices with a frequency > 100 Hz, so there are more than 100 packets per second. When a new client tried to connect to the service, we have found that the upgrade to the WebSocket transport blocked in the line where we were waiting for the outbound packet queue to become empty (i.e. a self.queue.join() call). This happened because the polling transport was not fast enough to drain the packet queue faster than it was filled from the other end by the measurement thread. (The WebSocket transport would have been fast enough, though).

The patch fixes the problem by storing outbound packets generated during the upgrade in a temporary "backlog" queue so the "real" outbound queue has a chance to become empty, allowing the transport upgrade to proceed. When the transport is upgraded to WebSocket, the backlog is flushed and then the backlog queue is deleted.

[miguelgrinberg]

Thanks. I will test this code carefully to make sure there are no problems. But just from reading the patch, does the backlog need to be a Queue? Can't you just use a simple list instead?

[ntamas]

I'm not sure and I just wanted to be on the safe side so I went for a Queue. Considering the fact that the websocket transport is available only when using eventlet (is that right?), a simple list should suffice because the queue won't be written from multiple threads. However, if you ever decide to implement websocket transport even for "ordinary", native threads, it seems to be more future-proof if we also use a queue there. I don't think that the difference is significant because the backlog would be used only during the transport upgrade, which should be relatively fast.

[ntamas]

Rebased this to the current master.

[miguelgrinberg]

I implemented a variant of your solution that uses a list. Thanks!