Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many docvalue_fields reported while there is less #63730

Closed
immon opened this issue Oct 15, 2020 · 2 comments · Fixed by #63806
Closed

Too many docvalue_fields reported while there is less #63730

immon opened this issue Oct 15, 2020 · 2 comments · Fixed by #63806
Assignees
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Comments

@immon
Copy link
Contributor

immon commented Oct 15, 2020

Elasticsearch version (bin/elasticsearch --version): 7.9.2

Plugins installed: []

JVM version (java -version): bundled

OS version (uname -a if on a Unix-like system): Elastic Cloud

Description of the problem including expected versus actual behavior:

Search query requesting more docvalue_fields than index configured setting max_docvalue_fields_search throws an error even if requested fields do not exist.

A non-existent field should not contribute to the limit.

Here is minimal reproduction:

PUT docvaluetest
{
  "settings": {
    "max_docvalue_fields_search": 1
  }
}

POST docvaluetest/_doc
{
  "@timestamp": "2020-10-15T11:11:11Z"
}

GET docvaluetest/_search
{
  "query": {
    "match_all": {}
  },
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    },
    {
      "field": "nonexistent1",
      "format": "date_time"
    }
  ]
}

Response:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Trying to retrieve too many docvalue_fields. Must be less than or equal to: [1] but was [2]. This limit can be set by changing the [index.max_docvalue_fields_search] index level setting."
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "docvaluetest",
        "node" : "YoeZKDjXS5mMBExIOWfgqA",
        "reason" : {
          "type" : "illegal_argument_exception",
          "reason" : "Trying to retrieve too many docvalue_fields. Must be less than or equal to: [1] but was [2]. This limit can be set by changing the [index.max_docvalue_fields_search] index level setting."
        }
      }
    ],
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "Trying to retrieve too many docvalue_fields. Must be less than or equal to: [1] but was [2]. This limit can be set by changing the [index.max_docvalue_fields_search] index level setting.",
      "caused_by" : {
        "type" : "illegal_argument_exception",
        "reason" : "Trying to retrieve too many docvalue_fields. Must be less than or equal to: [1] but was [2]. This limit can be set by changing the [index.max_docvalue_fields_search] index level setting."
      }
    }
  },
  "status" : 400
}

Expected behavior would be to first check field existence and only then verify the limits.

@immon immon added >bug needs:triage Requires assignment of a team area label labels Oct 15, 2020
@jtibshirani jtibshirani added :Search/Search Search-related issues that do not fall into other categories and removed needs:triage Requires assignment of a team area label labels Oct 15, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (:Search/Search)

@elasticmachine elasticmachine added the Team:Search Meta label for search team label Oct 15, 2020
@cbuescher cbuescher self-assigned this Oct 16, 2020
@cbuescher
Copy link
Member

I looked at the way we check the limit in FetchDocValuesContext and think we can check if the field is mapped for that particular index there.
I'll open a PR for discussion if this is the best way to fix this soon.

cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Oct 16, 2020
Currently we count every field requested in the search request bodies
'docvalue_fields' section towards the limit defined by
the 'max_docvalue_fields_search' index setting which defaults to 100. This can
be a problem e.g. if the user searches across several indices with some fields
present in one index but not the other and has to add the joint set of field
names to the query. We currently trip the limit even if the number of actually
mapped fields in each index is below the limit.
This change adds a step to distiguish between mappend and unmapped fields and
only count the former towards the limit.

Closes elastic#63730
cbuescher pushed a commit that referenced this issue Oct 21, 2020
Currently we count every field requested in the search request bodies
'docvalue_fields' section towards the limit defined by
the 'max_docvalue_fields_search' index setting which defaults to 100. This can
be a problem e.g. if the user searches across several indices with some fields
present in one index but not the other and has to add the joint set of field
names to the query. We currently trip the limit even if the number of actually
mapped fields in each index is below the limit.
This change adds a step to distiguish between mappend and unmapped fields and
only count the former towards the limit.

Closes #63730
cbuescher pushed a commit to cbuescher/elasticsearch that referenced this issue Oct 21, 2020
Currently we count every field requested in the search request bodies
'docvalue_fields' section towards the limit defined by the
'max_docvalue_fields_search' index setting which defaults to 100. This can be a
problem e.g. if the user searches across several indices with some fields
present in one index but not the other and has to add the joint set of field
names to the query. We currently trip the limit even if the number of actually
mapped fields in each index is below the limit.
This change adds a step to distiguish between mappend and unmapped fields and
only count the former towards the limit.

Closes elastic#63730
cbuescher pushed a commit that referenced this issue Oct 21, 2020
Currently we count every field requested in the search request bodies
'docvalue_fields' section towards the limit defined by the
'max_docvalue_fields_search' index setting which defaults to 100. This can be a
problem e.g. if the user searches across several indices with some fields
present in one index but not the other and has to add the joint set of field
names to the query. We currently trip the limit even if the number of actually
mapped fields in each index is below the limit.
This change adds a step to distiguish between mappend and unmapped fields and
only count the former towards the limit.

Closes #63730
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants