Cherry-pick #22389 to 7.x: Autodiscover ephemeral containers in kubernetes

[jsoriano]

Cherry-pick of PR #22389 to 7.x branch. Original message:

Enhancement: Autodiscover ephemeral containers in kubernetes when "scrape_ephemeral_containers" flag is set to True

What does this PR do?

Adds autodiscovery of kubernetes ephemeral containers. It is disabled by default and needs to set "scrape_ephemeral_containers" flag to "True"

Why is it important?

I'm using filebeat to scrape logs of pods in my organizations kubernetes clusters, and there is currently no support for collecting logs for ephemeral containers.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

1. Set kubernetes cluster with EphemeralContainers feature gate enabled
2. Run filebeat on kubernetes cluster with autodiscovery as in official docs.
3. Add config flag "scrape_ephemeral_containers: True"
4. Start ephemeral container and check if filebeat starts collecting logs

Related issues

• discussion https://discuss.elastic.co/t/kubernetes-autodiscovery-for-ephemeral-containers/253840

Use cases

Add support for autodiscovery of ephemeral containers in kubernetes cluster

Screenshots

Logs

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[elasticmachine]

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	16345
Skipped	1367
Total	17712

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: [Pull request #22434 opened]

• Start Time: 2020-11-04T17:40:17.695+0000

• Duration: 65 min 42 sec

Test stats 🧪

Test	Results
Failed	0
Passed	16345
Skipped	1367
Total	17712