Connection reset on an openshift cluster

[NicolasHoferer]

We are using Gunicorn + Flask on an Openshift cluster and are having issues when we try to do some load testing.

We use a single worker in Gunicorn and our application does inference using machine learning models.
Therefore, we cannot use multithreading/multiprocessing.

If we send one request after another for 5 minutes, we will not get into trouble, at least until today, where this also produces the error.
However, if we try to send more requests to the application (at most two in parallel), after some time we get the following error.
("Connection interrupted: ConnectionResetError(54, 'Connection reset by peer')", ConnectionResetError(54, 'Connection reset by peer')).

Then it stops working for about five seconds and works again until the error occurs again.

The requests we send to the application take between 0.1 seconds and about 10 seconds to be processed by the application (as said, some machine learning models are used).

To rule out ML and resource issues, we set up a route in Flask with no processing at all, just a 10-second sleep.
Now the error occurs again when more than one client is used to send requests.
Now that we are not using ML models, we can increase the number of threads from gunicorn.
For example, if we set --threads=3, the problem occurs only when we use 3 client threads to send requests.
So it looks like there is a problem that causes gunicorn to refuse to connect.
In the flasks and gunicorn logs we couldn't find any problems, even when debug is enabled.

Do you have any idea what the problem could be?
Perhaps Openshift is the problem, as we tried replicating the setup locally by running the app with Docker on one Mac and then sending requests from another Mac. In doing so, we had no issues with denied connections.

[benoitc]

no activity since awhile. closing feel free to create a new ticket if needed.