Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inner hits _source is now always wrapped in path to parent #26944

Closed
Mpdreamz opened this issue Oct 10, 2017 · 1 comment
Closed

Inner hits _source is now always wrapped in path to parent #26944

Mpdreamz opened this issue Oct 10, 2017 · 1 comment
Labels
:Search/Search Search-related issues that do not fall into other categories

Comments

@Mpdreamz
Copy link
Member

Prior to 6.x the _source of an inner hit was always rooted to the child object that you were referencing.

Given the following super simple document where children is nested

POST /sample/doc
{
  "children": [
    {
      "startTime": 37230000000,
      "endTime": 73840000000
    }
  ]
}

The inner hits for children used to look like this:

"_source" : {
  "startTime" : 37230000000,
  "endTime" : 73840000000
}

Now in 6.x the context of source has changed the root of the child. Note the additional children key. If it was N levels deep the actual source of the inner hit would be N levels deep too.

"_source" : {
  "children" : {
   "startTime" : 37230000000,
   "endTime" : 73840000000
  }
}

This changes a lot of code that assumes the current context of the inner hit is inner hit and not the root of the document.

Another side effect is that the returned inner hit _source json, that is now rooted to the original doc, can no longer be mapped back into the doc type because children no longer is an array.

So now _source neither maps back to the original doc nor child and always acts as if it was source filtered.

I spoke with @martijnvg and I realize this change relates to:

#23090
#16653

Which I am 100% behind, full paths should be specified always. I don't agree with the way these changes changed the way _source is presented back to user though.

cc @elastic/es-clients

@clintongormley
Copy link
Contributor

Just chatted to @martijnvg about this. It does feel more practical to return the nested _source without the prefix. Either way, it's a breaking change (which already landed in 6.0.0-beta2 in #26102). I really wouldn't want to break this again in 7.0. My inclination is to fix this in rc2.

/cc @mattweber

@martijnvg martijnvg removed the discuss label Oct 13, 2017
martijnvg added a commit that referenced this issue Oct 19, 2017
… its full path context

Due to a change happened via #26102 to make the nested source consistent
with or without source filtering, the _source of a nested inner hit was
always wrapped in the parent path. This turned out to be not ideal for
users relying on the nested source, as it would require additional parsing
on the client side. This change fixes this, the _source of nested inner hits
is now no longer wrapped by parent json objects, irregardless of whether
the _source is included as is or source filtering is used.

Internally source filtering and highlighting relies on the fact that the
_source of nested inner hits are accessible by its full field path, so
in order to now break this, the conversion of the _source into its binary
form is performed in FetchSourceSubPhase, after any potential source filtering
is performed to make sure the structure of _source of the nested inner hit
is consistent irregardless if source filtering is performed.

PR for #26944

Closes #26944
martijnvg added a commit that referenced this issue Oct 19, 2017
… its full path context

Due to a change happened via #26102 to make the nested source consistent
with or without source filtering, the _source of a nested inner hit was
always wrapped in the parent path. This turned out to be not ideal for
users relying on the nested source, as it would require additional parsing
on the client side. This change fixes this, the _source of nested inner hits
is now no longer wrapped by parent json objects, irregardless of whether
the _source is included as is or source filtering is used.

Internally source filtering and highlighting relies on the fact that the
_source of nested inner hits are accessible by its full field path, so
in order to now break this, the conversion of the _source into its binary
form is performed in FetchSourceSubPhase, after any potential source filtering
is performed to make sure the structure of _source of the nested inner hit
is consistent irregardless if source filtering is performed.

PR for #26944

Closes #26944
@clintongormley clintongormley added :Search/Search Search-related issues that do not fall into other categories and removed :Nested Docs labels Feb 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

No branches or pull requests

4 participants