Cherry-pick #22189 to 7.x: Kubernetes metadata enhancements

[ChrsMark]

Cherry-pick of PR #22189 to 7.x branch. Original message:

What does this PR do?

1. adds kubernetes.host.hostname on node metadata,
2. attach node metadata on pod metadata by default (without add_resource_metadata required to enabled) for both events coming from autodiscovered pods but also for events enriched by add_kubernetes_metadata
3. attach namespace metadata on pod metadata by default (without add_resource_metadata required to enabled) for both events coming from autodiscovered pods but also for events enriched by add_kubernetes_metadata
4. Enables always add_resource_metadata section for autodiscover and add_kubernetes_metadata.

Why is it important?

This information is useful in various ways. See more on the respective issue: #20434

How to test this PR manually

1. Use autodiscover config like:

Test with autodiscover provider

filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*${data.kubernetes.container.id}.log

2. Make sure that Redis module is enabled and that kubernetes.node.hostname field is populated.

Test with add_kubernetes_metadata processor

filebeat.inputs:
- type: container
  paths:
    - /var/log/containers/*.log
  processors:
    - add_kubernetes_metadata:
        host: ${NODE_NAME}
        matchers:
        - logs_path:
            logs_path: "/var/log/containers/"

2. Make sure that kubernetes.node.hostname field is populated as well as kubernetes.node.* and kubernetes.namespace.*.

Related issues

• Closes Add kubernetes.node.hostname field with the actual hostname of the node #20434

Screenshots

Filebeat & add_kubernetes_metadata:

Autodiscover:

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[elasticmachine]

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	16368
Skipped	1368
Total	17736

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: [Started by user Chris Mark, Replayed #1]

• Start Time: 2020-11-12T09:33:31.225+0000

• Duration: 70 min 0 sec

Test stats 🧪

Test	Results
Failed	0
Passed	16368
Skipped	1368
Total	17736