Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

append_fields in template leads to invalid template if parent key is already contained #1959

Closed
simitt opened this issue Feb 27, 2019 · 3 comments
Assignees

Comments

@simitt
Copy link
Contributor

simitt commented Feb 27, 2019

In case additional fields should be added to the ES index template, one can use the experimental setting setup.template.append_fields in the apm-server.yml.
The setting cannot be used to change the type of an existing field, but to add new fields. If the field is already contained in the mapping an error is raised.

As @axw reported adding context.request.socket.remote_address raises an error, although the field is not contained in the template. Some investigations suggest that there is a bug in the implementation using field.HasNode, which does not allow to add additional keys under an existing leaf, even if it is of type object.

@simitt
Copy link
Contributor Author

simitt commented Feb 27, 2019

There's a PR up in libbeat elastic/beats#10959, that would need to be backported to >= 6.6.

The bug currently only becomes obvious in 6.6, as we added non indexed fields to the fields.yml file for documentation purposes. As soon as those fields are part of the template, the current implementation of setup.template.append_fields does not allow to add further keys under a leaf node, even when it is of type object. The PR in beats aims to fix this.

From > 6.6 on we removed the non indexed fields from the template again (when reverting the aliases). @graphaelli I am not sure whether or not the bugfix in libbeat is going to be backport to 6.6. I suggest that depending on the backports we either add the non indexed fields also from > 6.6 on or we remove them again from 6.6 in the next patch.

@graphaelli
Copy link
Member

That sounds like a good plan.

@simitt
Copy link
Contributor Author

simitt commented Mar 11, 2019

follow up issue for fixing documentation around non indexed fields #1980.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants