docs: Finish Application logging docs (#2542) (#2558)

* docs: follow-ups to app logs

* docs: add log correlation info to plaintext

* docs: add service.name to tabbed widget

(cherry picked from commit 47e0adf)

Co-authored-by: Brandon Morelli <[email protected]>

docs/en/observability/application-logs.asciidoc

@@ -36,25 +36,26 @@ For example, the same example shown above might look like this when structured w
 [[log-correlation]]
 == Log correlation
 
+To correlate your logs, you must annotate your logs with the APM identifier, `service.name`.
+Elastic APM integrates with the most popular logging frameworks in each programming language to accomplish this automatically.
+
+// tag::correlate-logs[]
 Correlating your application logs with trace events allows you to:
 
 * view the context of a log and the parameters a user provided
 * view all logs belonging to a particular trace
 * easily move between logs and traces when debugging application issues in {kib}
 
-To correlate your logs, you must add APM identifiers,
-like trace, transaction, and error ID fields into your application's logs.
-Elastic APM integrates with the most popular logging frameworks in each programming language to accomplish this automatically.
-
-Learn more about log correlation in the APM Guide: {apm-guide-ref}/log-correlation.html[log correlation]
-Or in any of the agent-specific ingestion guides:
+Learn more about log correlation in the APM Guide: {apm-guide-ref}/log-correlation.html[log correlation],
+or in any of the agent-specific ingestion guides:
 
 * {apm-go-ref}/log-correlation.html[Go]
 * {apm-java-ref}/log-correlation.html[Java]
 * {apm-dotnet-ref}/log-correlation.html[.NET]
 * {apm-node-ref}/log-correlation.html[Node.js]
 * {apm-py-ref}/log-correlation.html[Python]
 * {apm-ruby-ref}/log-correlation.html[Ruby]
+// end::correlate-logs[]
 
 [float]
 [[ingest-application-logs]]
@@ -213,6 +214,16 @@ To enable these features, you'll need to parse the contents of your logs into EC
 To learn how to use the Grok processor to parse application logs before indexing,
 see {ref}/common-log-format-example.html[Example: Parse logs in the Common Log Format].
 
+[float]
+=== Step 3: Correlate your logs
+
+include::application-logs.asciidoc[tag=correlate-logs]
+
+[float]
+=== Step 4: View your logs in {kib}
+
+Use the <<apm,APM>> or <<monitor-logs,Logs>> UI to search, filter, and visualize your logs.
+
 [float]
 === Learn more
 
@@ -259,6 +270,12 @@ _See the Ruby or Java agent docs._
 include::./tab-widgets/filebeat-logs/widget.asciidoc[]
 :!ecs-logs:
 
+[float]
+=== Step 3: View your logs in {kib}
+
+Use the <<apm,APM>> or <<monitor-logs,Logs>> UI to search, filter, and visualize your logs.
+
+
 [[apm-agent-log-sending]]
 == APM agent log sending
 

docs/en/observability/tab-widgets/filebeat-logs/content.asciidoc

@@ -14,18 +14,24 @@ filebeat.inputs:
   paths: /path/to/logs.json
   parsers:
     - ndjson:
-      keys_under_root: true
-      overwrite_keys: true
-      add_error_key: true
-      expand_keys: true
+      overwrite_keys: true <2>
+      add_error_key: true <3>
+      expand_keys: true <4>
+  fields:
+    service.name: your_service_name <5>
 
-processors:
+processors: <6>
   - add_host_metadata: ~
   - add_cloud_metadata: ~
   - add_docker_metadata: ~
   - add_kubernetes_metadata: ~
 ----
 <1> Use the filestream input to read lines from active log files.
+<2> Values from the decoded JSON object overwrite the fields that {filebeat} normally adds (type, source, offset, etc.) in case of conflicts.
+<3> {filebeat} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors.
+<4> {filebeat} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure.
+<5> The `service.name` of the service you're collecting logs from. This is required for <<log-correlation>>.
+<6> Processors enhance your data. See {filebeat-ref}/filtering-and-enhancing-data.html[processors] to learn more.
 endif::ecs-logs[]
 ifdef::plaintext[]
 . Configure filebeat.yaml file to start collecting log data.
@@ -37,8 +43,10 @@ ifdef::plaintext[]
 ----
 filebeat.inputs:
 - type: filestream <1>
-  paths: /path/to/logs.log
+  paths: /path/to/logs.log <2>
 ----
+<1> Reads lines from an active log file.
+<2> A list of glob-based paths that will be crawled and fetched.
 endif::plaintext[]
 
 // end::logs[]
@@ -57,11 +65,13 @@ ifdef::ecs-logs[]
 [source,yaml]
 ----
 annotations:
-  co.elastic.logs/json.keys_under_root: true
-  co.elastic.logs/json.overwrite_keys: true
-  co.elastic.logs/json.add_error_key: true
-  co.elastic.logs/json.expand_keys: true
+  co.elastic.logs/json.overwrite_keys: true <1>
+  co.elastic.logs/json.add_error_key: true <2>
+  co.elastic.logs/json.expand_keys: true <3>
 ----
+<1> Values from the decoded JSON object overwrite the fields that {filebeat} normally adds (type, source, offset, etc.) in case of conflicts.
+<2> {filebeat} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors.
+<3> {filebeat} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure.
 endif::ecs-logs[]
 ifdef::plaintext[]
 endif::plaintext[]
@@ -82,11 +92,13 @@ ifdef::ecs-logs[]
 .docker-compose.yml
 ----
 labels:
-  co.elastic.logs/json.keys_under_root: true
-  co.elastic.logs/json.overwrite_keys: true
-  co.elastic.logs/json.add_error_key: true
-  co.elastic.logs/json.expand_keys: true
+  co.elastic.logs/json.overwrite_keys: true <1>
+  co.elastic.logs/json.add_error_key: true <2>
+  co.elastic.logs/json.expand_keys: true <3>
 ----
+<1> Values from the decoded JSON object overwrite the fields that {filebeat} normally adds (type, source, offset, etc.) in case of conflicts.
+<2> {filebeat} adds an "error.message" and "error.type: json" key in case of JSON unmarshalling errors.
+<3> {filebeat} will recursively de-dot keys in the decoded JSON, and expand them into a hierarchical object structure.
 endif::ecs-logs[]
 ifdef::plaintext[]
 endif::plaintext[]