Kibana should cancel previous requests before initiating new request from same session

[lanerjo]

Describe the feature:
Per elastic/elasticsearch#23250 tasks can be killed

Kibana should be able to generate and log an Opaque-Id per #16493 for each user request

Kibana in order to keep from sending too many queries from a single user to Elasticsearch should cancel the previously generated query based on the Opaque-Id before (or simultaneously) sending another request from the same session.

Description of the problem including expected versus actual behavior:
User starts to load any page in Kibana
Page sends Query to Elasticsearch which may take some time to respond
User Decides, maybe I want another page | index pattern | plugin | dashboard etc...
User makes change before the previous request has been completed in Elasticsearch

Kibana should request to cancel the first task which would then alleviate the unnecessary load on Elasticsearch, before making the new request.

[elasticmachine]

Pinging @elastic/kibana-platform

[epixa]

Core can't really tell when one ES request versus some other ES request should be canceled when the URL changes. Many ES queries should survive navigation. It can't even necessarily tell when many types of URL changes are navigation related since apps can do basically whatever they want in that event. So there's no generic mechanism that could be applied universally to cancel queries based on some UI operation.

I think this is something that should probably be built into the new search abstraction that the app arch team is working on. Then any app that is using the search abstraction could invoke the kill operation on the service whenever it did internal navigation that warranted it or even when the entire app is unmounted.

[lanerjo]

I am thinking about this from a User experience. I cannot recall anytime when using Kibana, that I would need a call to complete if I have navigated or changed some parameter. In fact the returning results cannot be given to the user if the user has changed any of the previous request.

Examples:

1. Discover tab, defaults to some index pattern, I change the index pattern before the initial load, the first call is not needed (thus a waste of resources).
2. Timelion, I make any change to the query and it is sent, before getting results back, I change the query again. The original results are no longer needed and the request should be cancelled since they cannot be displayed.
3. Visualization, see timelion above
4. I load a dashboard, quickly realize that is not what I wanted, so I load a different dashboard. Again the results from the first request will never be displayed and the task should be killed
5. Etc, etc, etc

Core should keep some state based on either user or session. - This would allow for tracking responses from ES. There has to be some form of state in order to keep the right request going to the right user. This could possibly be expanded to keep track of responses that have been completed, and if a response has not been completed and a navigation is performed, or the previous request is no longer useful, then cancel the previous request.

This would take some considerable thought on implementation, but would likely help to get rid of the nickname for Kibana of "Killbana"

[epixa]

All of the examples you mention are indeed cases where you'd want the requests to be terminated, and I also think they'd all be covered with the search abstraction that I mentioned in my previous comment.

Some examples of requests that should not be terminated regardless of navigation state:

1. Refreshing an expired access token
2. Polling to check the status on a global background task (e.g. report generation)
3. Retrieving updated license information

Not all requests in Kibana are related to the navigational state, so we can't have a universal system tied to navigation that can handle termination of all requests. To make things more complicated, there isn't a global concept of navigation at all. Each application can define its own mechanisms for URL/navigation handling without core having any knowledge of it.

The unifying factor of most of the scenarios where we'd definitely want to cancel requests based on some sort of user action like navigation is that they're generally about searching the user's data. The data/search service is what those apps would use to make their requests, so it makes sense that it would also be used to cancel them.

[lukeelmers]

I think most of the concrete examples in this issue would be already covered by #17845, which is part of the larger project to rewrite our querying infrastructure that @epixa is referring to.

For now I'll close this as a duplicate -- but @lanerjo feel free to respond if you feel those issues I linked to are missing any of your use cases.

cc @elastic/kibana-app-arch