-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kibana generates requests with too many docvalue_fields, causing Elasticsearch to throw errors #22897
Comments
is there a way to stop this behavior at the Kibana layer -- a date field doesn't mean we need to send a doc value request for it |
A short explanation here, why this is happening: Elasticsearch accepts a potential way wider range on date formats that you can inject in a document that you insert, than Kibana can in the end parse and apply it's date field formatter on. That's why we can't use the date fields from _source, since they would potentially be unparseable by us. So we request them as docvalues, since we can make sure that the format returned is parseable by us. Since you can expand every document in discover also, and we'll need to apply the date field formatters to what's shown, we need to load all those doc_values. The only thing I could see us preventing this, if we would make a second request for the full document once it's expended in Discover, and only load the fields shown in the table before that (and thus only the date fields shown as columns via docvalues). That has the downside of having quiet some overhead every time you want to expand a document in discover, and from our experience users are navigating rather quickly through different documents in Discover, which wouldn't be good possible anymore in that case. Maybe we could in the future add this as a configuration option whether you want the one or the other behavior (everything loaded immediately, or just the data shown in the table and full doc only loaded once expanded). cc @kertal |
Pinging @elastic/kibana-app-arch (Team:AppArch) |
@ppisljar @lukasolson Just got a report about this as a bug from Alona, and it turns out that we are still requesting docvalue_fields even on aggregated requests with size 0. The problem is that there are some indexes with >100 date fields, and this will cause requests to fail. |
thanks @wylieconlon, we'll try to prioritize fixing this |
Im seeing this same problem in Kibana 7.9.3 with filebeat indexes. request: response |
@omarmarquez It should be fixed in 7.11 as per elastic/elasticsearch#63730 As a workaround update filebeat indices as following:
|
This should be resolved as a side effect of #82383, where we switched to requesting these date fields via the search fields API instead of The fields API does not have the limits that |
Describe the feature:
Kibana automatically queries all date fields in the index patterns as doc_values. There should be a way to choose which fields it requests as doc values. Or, better, it should request as doc values only fields actually relevant to the query.
Describe a specific use case for the feature:
This request has context coming from https://discuss.elastic.co/t/kibana-requesting-too-many-doc-values/147760
We have an elasticsearch cluster that indexes some events that flow through our system for debugging use. These events have pretty widely varying formats, so they end up generating a lot of different fields. Kibana maps about 2000 fields for the indexes. We don't configure these indexes manually, and just let Elasticsearch automatically generate indexes based on the data.
Performance has never been a problem. Type conflicts are pretty rare and haven't been problematic enough to warrant any action.
One day, I refreshed field mappings, and all search queries are breaking with the error:
It looks like the indexes now have 101 different date type fields. Kibana seems to automatically request every date fields as docvalue fields in every single request.
These are for "Discover" requests, and we don't ever sort/aggregate on any of these fields. I've actually never had a need sort/aggregate on any of these date fields.
Rather than having to go and manually map these to strings, Kibana should be clever enough to not request them as doc values by examining the query. If that's not possible, there should at least be some way to configure the index template to restrict which fields are used as doc values.
The text was updated successfully, but these errors were encountered: