Server takes too long to respond

[jc-harrison]

The first time an EventsTablesUnion object is constructed, all events tables for all available dates are "added to cache" (i.e. the query IDs for the corresponding Table objects are added to cache.cached, and the query state is set to completed in redis). This process can take several seconds.

As query object construction in the FlowMachine server happens on the main thread, before creating a separate thread to run the query, this is a blocking operation and causes the server to wait several seconds before responding. In some cases (e.g when building the docs without running the pre-cache script) this wait is too long and results in FlowClient raising:

FlowclientConnectionError: Unable to connect to FlowKit API at http://localhost:9090: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[greenape]

This is actually pretty serious I think - Hypercorn's default timeout is, I believe, 5s.

At present, while the flowmachine server is async it largely operates serially - so that means that if your request is in a queue which is going to take more than 5s to get through, you'll error out.

In principle, adding a sleep(4) inside Query's __init__ for example, and then firing off two distinct run query requests should make the second fail, and will definitely make any third request fail.

[greenape]

So that gives us two issues:

1. Query object creation can be pretty slow
2. Even if query object creation is pretty fast, it blocks the server from responding, so even if it took 1s on average you'd hit this issue with quite a small number of unique run query requests close together.