diff --git a/filebeat/docs/running-on-kubernetes.asciidoc b/filebeat/docs/running-on-kubernetes.asciidoc index 40c18b3f8f2b..bab7de20b589 100644 --- a/filebeat/docs/running-on-kubernetes.asciidoc +++ b/filebeat/docs/running-on-kubernetes.asciidoc @@ -22,10 +22,10 @@ to ensure there's a running instance on each node of the cluster. The Docker logs host folder (`/var/lib/docker/containers`) is mounted on the {beatname_uc} container. {beatname_uc} starts an input for the files and -begins harvesting them as soon as they appear in the folder. +begins harvesting them as soon as they appear in the folder. Everything is deployed under the `kube-system` namespace by default. To change -the namespace, modify the manifest file. +the namespace, modify the manifest file. To download the manifest file, run: @@ -110,7 +110,7 @@ oc patch namespace kube-system -p \ ---- + This command sets the node selector for the project to an empty string. If you -don't run this command, the default node selector will skip master nodes. +don't run this command, the default node selector will skip master nodes. [float] @@ -135,3 +135,79 @@ filebeat 32 32 0 32 0 Log events should start flowing to Elasticsearch. The events are annotated with metadata added by the <> processor. + + +[float] +==== Parsing json logs + +It is common case when collecting logs from workloads running on Kubernetes that these +applications are logging in json format. In these case, special handling can be applied so as to +parse these json logs properly and decode them into fields. Bellow there are provided 2 different ways +of configuring <> so as to identify and parse json logs. +We will use an example of one Pod with 2 containers where only one of these logs in json format. + +Example log: +``` +{"type":"log","@timestamp":"2020-11-16T14:30:13+00:00","tags":["warning","plugins","licensing"],"pid":7,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"} +``` + + +. Using `json.*` options with templates ++ +[source,yaml] +------------------------------------------------ +filebeat.autodiscover: + providers: + - type: kubernetes + node: ${NODE_NAME} + templates: + - condition: + contains: + kubernetes.container.name: "no-json-logging" + config: + - type: container + paths: + - "/var/log/containers/*-${data.kubernetes.container.id}.log" + - condition: + contains: + kubernetes.container.name: "json-logging" + config: + - type: container + paths: + - "/var/log/containers/*-${data.kubernetes.container.id}.log" + json.keys_under_root: true + json.add_error_key: true + json.message_key: message +------------------------------------------------ + +. Using `json.*` options with hints ++ +Key part here is to properly annotate the Pod to only parse logs of the correct container +as json logs. In this, annotation should be constructed like this: ++ +`co.elastic.logs./json.keys_under_root: "true"` ++ +Autodiscovery configuration: ++ +[source,yaml] +------------------------------------------------ +filebeat.autodiscover: + providers: + - type: kubernetes + node: ${NODE_NAME} + hints.enabled: true + hints.default_config: + type: container + paths: + - /var/log/containers/*${data.kubernetes.container.id}.log +------------------------------------------------ ++ +Then annotate the pod properly: ++ +[source,yaml] +------------------------------------------------ +annotations: + co.elastic.logs.json-logging/json.keys_under_root: "true" + co.elastic.logs.json-logging/json.add_error_key: "true" + co.elastic.logs.json-logging/json.message_key: "message" +------------------------------------------------