Cherry-pick #18223 to 7.x: Disable host fields for "cloud", panw, cef modules

[andrewkroh]

Cherry-pick of PR #18223 to 7.x branch. Original message:

What does this PR do?

This changes the default configuration of Filebeat to not add host fields to events that
originated in other places. The host field is defined in ECS as "host on which the event happened" but for data pulled from cloud APIs for data forwarded to Filebeat from other sources (PANW, CEF) this host field is inaccurate.

The affected "cloud" modules are azure, aws, googlecloud, o365, and okta. By default they will
tag events with cloud. This causes the module to not add host.name at the input state. And then the default configuration for Filebeat was updated to add a when condition to the add_host_metadata processors to skip events containing the cloud tag.

For PANW and CEF when data is forwarded to Filebeat from another host/device (this is most of the time) you don't want Filebeat to add host. So by default this modules add a forwarded tag to events that behaves the same as the cloud tag. If you configure the module to not include the forwarded tag (e.g. var.tags: [my_tag]) then Filebeat will add the host.* fields.

And for PANW I added some additional static observer.* fields.

Why is it important?

We want Filebeat to follow Elastic Common Schema. And setting host with the correct value is part of that. By setting (or not setting host) we can better interpret events. Without this change the Filebeat host is being attributed as the source of many cloud based audit/login events.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Run this config from the x-pack/filebeat dir and verify that events do not contain host.

filebeat.modules:
- module: panw
  panos:
    var.input: file
    var.paths:
      - module/panw/panos/test/*.log
processors:
  - add_host_metadata:
      when.not.or:
        - contains.tags: cloud
        - contains.tags: forwarded
output.console.pretty: true

Related issues

Relates: #13920
Requires: #18159

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #19074 updated]

• Start Time: 2020-06-09T16:44:50.237+0000

• Duration: 66 min 34 sec

Test stats 🧪

Test	Results
Failed	0
Passed	2656
Skipped	417
Total	3073