Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add json parsing doc section for Filebeat on Kubernetes #22621

Merged
merged 4 commits into from
Nov 18, 2020

Conversation

ChrsMark
Copy link
Member

@ChrsMark ChrsMark commented Nov 17, 2020

Adds documentation section about json parsing for Filebeat on Kubernetes.

Closes #22516

@dedemorton do you think I should add it in the new docs guide we are preparing?

@ChrsMark ChrsMark added docs Team:Platforms Label for the Integrations - Platforms team labels Nov 17, 2020
@ChrsMark ChrsMark self-assigned this Nov 17, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 17, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Nov 17, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #22621 updated]

  • Start Time: 2020-11-17T15:24:58.568+0000

  • Duration: 9 min 58 sec

❕ Flaky test report

No test was executed to be analysed.

@dede
Copy link

dede commented Nov 17, 2020

Hey @ChrsMark, i think you wanted to mention @dedemorton 😃

@ChrsMark
Copy link
Member Author

Hey @ChrsMark, i think you wanted to mention @dedemorton 😃

🤦🏼 you r right! sorry for this!

Signed-off-by: chrismark <[email protected]>
Signed-off-by: chrismark <[email protected]>
paths:
- "/var/log/containers/*-${data.kubernetes.container.id}.log"
processors:
- decode_json_fields:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this one still be json.*?


It is common case when collecting logs from workloads running on Kubernetes that these
applications are logging in json format. In these case, special handling can be applied so as to
parse these json logs properly and decode them into fields. Bellow there are provided 3 different ways
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be more opinionated here, adding just 2 options. One for template based autodiscover and another one for hints based autodiscover. I would go for json.* as it gives the user more freedom down the line (multiline/line filtering happens after JSON decoding).

Signed-off-by: chrismark <[email protected]>
@ChrsMark
Copy link
Member Author

@exekias fixed, could you have another look please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Team:Platforms Label for the Integrations - Platforms team v7.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enrich Filebeat on k8s docs with json parsing settings
4 participants