Skip to content

Commit

Permalink
Add json parsing doc section for Filebeat on Kubernetes (elastic#22621)
Browse files Browse the repository at this point in the history
  • Loading branch information
ChrsMark authored Nov 18, 2020
1 parent b987bb6 commit b0da5cb
Showing 1 changed file with 79 additions and 3 deletions.
82 changes: 79 additions & 3 deletions filebeat/docs/running-on-kubernetes.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ to ensure there's a running instance on each node of the cluster.

The Docker logs host folder (`/var/lib/docker/containers`) is mounted on the
{beatname_uc} container. {beatname_uc} starts an input for the files and
begins harvesting them as soon as they appear in the folder.
begins harvesting them as soon as they appear in the folder.

Everything is deployed under the `kube-system` namespace by default. To change
the namespace, modify the manifest file.
the namespace, modify the manifest file.

To download the manifest file, run:

Expand Down Expand Up @@ -110,7 +110,7 @@ oc patch namespace kube-system -p \
----
+
This command sets the node selector for the project to an empty string. If you
don't run this command, the default node selector will skip master nodes.
don't run this command, the default node selector will skip master nodes.


[float]
Expand All @@ -135,3 +135,79 @@ filebeat 32 32 0 32 0 <none>

Log events should start flowing to Elasticsearch. The events are annotated with
metadata added by the <<add-kubernetes-metadata>> processor.


[float]
==== Parsing json logs

It is common case when collecting logs from workloads running on Kubernetes that these
applications are logging in json format. In these case, special handling can be applied so as to
parse these json logs properly and decode them into fields. Bellow there are provided 2 different ways
of configuring <<configuration-autodiscover, filebeat's autodiscover>> so as to identify and parse json logs.
We will use an example of one Pod with 2 containers where only one of these logs in json format.

Example log:
```
{"type":"log","@timestamp":"2020-11-16T14:30:13+00:00","tags":["warning","plugins","licensing"],"pid":7,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
```


. Using `json.*` options with templates
+
[source,yaml]
------------------------------------------------
filebeat.autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}
templates:
- condition:
contains:
kubernetes.container.name: "no-json-logging"
config:
- type: container
paths:
- "/var/log/containers/*-${data.kubernetes.container.id}.log"
- condition:
contains:
kubernetes.container.name: "json-logging"
config:
- type: container
paths:
- "/var/log/containers/*-${data.kubernetes.container.id}.log"
json.keys_under_root: true
json.add_error_key: true
json.message_key: message
------------------------------------------------

. Using `json.*` options with hints
+
Key part here is to properly annotate the Pod to only parse logs of the correct container
as json logs. In this, annotation should be constructed like this:
+
`co.elastic.logs.<container_name>/json.keys_under_root: "true"`
+
Autodiscovery configuration:
+
[source,yaml]
------------------------------------------------
filebeat.autodiscover:
providers:
- type: kubernetes
node: ${NODE_NAME}
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/log/containers/*${data.kubernetes.container.id}.log
------------------------------------------------
+
Then annotate the pod properly:
+
[source,yaml]
------------------------------------------------
annotations:
co.elastic.logs.json-logging/json.keys_under_root: "true"
co.elastic.logs.json-logging/json.add_error_key: "true"
co.elastic.logs.json-logging/json.message_key: "message"
------------------------------------------------

0 comments on commit b0da5cb

Please sign in to comment.