Reconnect on HTTP 503

[turicfr]

Hello,

First of all, thank you for developing this amazing library!

I have a client that needs to be connected continuously. However, I encountred a server error and it responded with HTTP 503 for a ping packet. Here are the logs:

[DEBUG] urllib3.connectionpool: https://chat.wikia-services.com:443 "GET /socket.io/?key=Chat%3A%3Acookies%3A%3A62512b1187947aab9501d9c8def38c15b482b51a&roomId=25286&serverId=1534080&transport=polling&EIO=3&sid=sRmxpOX6WgrS5hHLAAUu&t=1589350146.86141 HTTP/1.1" 200 3
[INFO] engineio.client: Received packet PONG data None
[INFO] engineio.client: Sending polling GET request to https://chat.wikia-services.com/socket.io/?key=Chat%3A%3Acookies%3A%3A62512b1187947aab9501d9c8def38c15b482b51a&roomId=25286&serverId=1534080&transport=polling&EIO=3&sid=sRmxpOX6WgrS5hHLAAUu
[INFO] engineio.client: Sending packet PING data None
[DEBUG] urllib3.connectionpool: https://chat.wikia-services.com:443 "POST /socket.io/?key=Chat%3A%3Acookies%3A%3A62512b1187947aab9501d9c8def38c15b482b51a&roomId=25286&serverId=1534080&transport=polling&EIO=3&sid=sRmxpOX6WgrS5hHLAAUu HTTP/1.1" 503 286
[WARNING] engineio.client: Unexpected status code 503 in server response, aborting
[INFO] engineio.client: Exiting write loop task
[INFO] engineio.client: Exiting ping task
[INFO] engineio.client: HTTP GET request to https://chat.wikia-services.com/socket.io/?key=Chat%3A%3Acookies%3A%3A62512b1187947aab9501d9c8def38c15b482b51a&roomId=25286&serverId=1534080&transport=polling&EIO=3&sid=sRmxpOX6WgrS5hHLAAUu&t=1589350156.8699625 failed with error HTTPSConnectionPool(host='chat.wikia-services.com', port=443): Read timed out. (read timeout=25.0).
[WARNING] engineio.client: Connection refused by the server, aborting
[INFO] engineio.client: Waiting for write loop task to end
[INFO] engineio.client: Waiting for ping loop task to end
[INFO] engineio.client: Exiting read loop task

As you can see the client simply exits without trying to reconnect. Is there a way to let it reconnect?

I'm using python-socketio 4.5.1 and python-engineio 3.12.1.

[miguelgrinberg]

There is nothing specific on the HTTP response codes. Any response status outside of the 200-399 range should trigger a disconnect. Do you have some example code I can use to reproduce this problem?

[turicfr]

Thanks for the fast response!

I'm able to reproduce the problem with the following example client and server:

import socketio

sio = socketio.Client(logger=True, engineio_logger=True)
sio.connect("http://localhost:8000")

import socketio
import eventlet

sio = socketio.Server(logger=True, engineio_logger=True, ping_interval=5, allow_upgrades=False)
app = socketio.WSGIApp(sio)

count = 0

old_handle_request = sio.handle_request
def new_handle_request(environ, start_response):
    print("received request")
    global count
    count += 1
    if count >= 5:
        raise Exception
    return old_handle_request(environ, start_response)
sio.handle_request = new_handle_request

eventlet.wsgi.server(eventlet.listen(("", 8000)), app)

And here are the logs:

Attempting polling connection to http://localhost:8000/socket.io/?transport=polling&EIO=3
Polling connection accepted with {'sid': '6b37e9ab1118406ba51044b25d8e6f13', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 5000}
Engine.IO connection established
Received packet MESSAGE data 0
Namespace / is connected
Sending packet PING data None
Sending polling GET request to http://localhost:8000/socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13
Received packet PONG data None
Sending polling GET request to http://localhost:8000/socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13
Sending packet PING data None
Unexpected status code 500 in server response, aborting
Exiting write loop task
Exiting ping task
Waiting for write loop task to end
Waiting for ping loop task to end
Exiting read loop task

Server initialized for eventlet.
(4452) wsgi starting up on http://0.0.0.0:8000
(4452) accepted ('127.0.0.1', 18565)
received request
6b37e9ab1118406ba51044b25d8e6f13: Sending packet OPEN data {'sid': '6b37e9ab1118406ba51044b25d8e6f13', 'upgrades': [], 'pingTimeout': 60000, 'pingInterval': 5000}
6b37e9ab1118406ba51044b25d8e6f13: Sending packet MESSAGE data 0
127.0.0.1 - - [15/May/2020 17:28:05] "GET /socket.io/?transport=polling&EIO=3&t=1589552883.667176 HTTP/1.1" 200 336 0.000000
received request
6b37e9ab1118406ba51044b25d8e6f13: Received packet PING data None
6b37e9ab1118406ba51044b25d8e6f13: Sending packet PONG data None
127.0.0.1 - - [15/May/2020 17:28:05] "POST /socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13 HTTP/1.1" 200 167 0.015628
(4452) accepted ('127.0.0.1', 18568)
received request
127.0.0.1 - - [15/May/2020 17:28:07] "GET /socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13&t=1589552885.7448165 HTTP/1.1" 200 183 0.000000
received request
received request
Traceback (most recent call last):
  File "D:\Program Files\Python38\lib\site-packages\eventlet\wsgi.py", line 566, in handle_one_response
    result = self.application(self.environ, start_response)
  File "D:\Program Files\Python38\lib\site-packages\engineio\middleware.py", line 60, in __call__
    return self.engineio_app.handle_request(environ, start_response)
  File "server_test.py", line 15, in new_handle_request
    raise Exception
Exception

127.0.0.1 - - [15/May/2020 17:28:10] "POST /socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13 HTTP/1.1" 500 593 0.000000
6b37e9ab1118406ba51044b25d8e6f13: Client is gone, closing socket
6b37e9ab1118406ba51044b25d8e6f13: Client is gone, closing socket
127.0.0.1 - - [15/May/2020 17:29:05] "GET /socket.io/?transport=polling&EIO=3&sid=6b37e9ab1118406ba51044b25d8e6f13&t=1589552887.8068392 HTTP/1.1" 200 179 57.923627

In this example, the server responds with 500 after a while. The client exits and doesn't try to reconnect.

[turicfr]

Any progress on this?

[miguelgrinberg]

No, I haven't had time to look into this issue yet. I consider it low priority because if the server responds with a 500 it indicates a bug in the server that should be fixed. Also I haven't tested this, but I believe there is a simple workaround that can be used in the meantime:

while True:
    sio.connect("http://localhost:8000")
    sio.sleep(5)

I will eventually get to it, but I have a couple other issues that I have prioritized above this one that I need to solve first.

[miguelgrinberg]

This issue should be addressed by miguelgrinberg/python-engineio#254, to be released with python-engineio 4.9.0.