Match by container id by default in add_kubernetes_metadata

[jsoriano]

Describe the enhancement:

Since 7.7, add_process_metadata can use PIDs of processes running in containers to add the container.id field.
From the container.id, add_kubernetes_metadata can use the container indexer and the fields matcher to enrich the event with kubernetes fields.
This gives a lot of possibilities in Auditbeat, where auditd events contain process information, we could make it the default indexers and matchers in Auditbeat. We can consider doing it in other beats too.

Describe a specific use case for the enhancement or feature:

Ease setup of Auditbeat in Kubernetes (see https://github.com/elastic/beats/pull/17431/files for an example configuration to do it now).

Related to #9668.

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[danmx]

at the same time it would be great to decouple metricbeat components from auditbeat:
https://github.com/elastic/beats/blob/master/auditbeat/cmd/root.go#L27-L28

[jsoriano]

at the same time it would be great to decouple metricbeat components from auditbeat:
https://github.com/elastic/beats/blob/master/auditbeat/cmd/root.go#L27-L28

This would be a completely separate discussion 😃 Is there any reason why you would like to see them decoupled? This coupling comes because Auditbeat reuses a lot of Metricbeat code. Auditbeat could be seen as a Metricbeat but with different modules. Having them as completely separate codebases would lead to lots of duplications in code, in testing...

[danmx]

One reason for the split could be that it's confusticating for new contributors. When I started to analyse the code I expected to see all common things in libbeats and beat-specific in their corresponding directories. I was surprised when I needed to go through another beat's code to debug issues or understand certain functionalities.

Another thing could be cleaner dependency tree and codebase. Common (more than one beat) functionalities would go to libbeat and beats pick and choose what functionalities they use (that would fit with #8199 and adding osquery module to Auditbeat for more robust security tool).

[jsoriano]

Yes, it can make sense to move all the common code to libbeat, but this would be a big refactor at this point, and not related to this issue.

adding osquery module to Auditbeat for more robust security tool

This is also a different discussion 🙂 Take into account that multiple Beats can be combined as a single security solution, and this is not going to change. For example logs are going to continue being collected by Filebeat.

[danmx]

Yes, it can make sense to move all the common code to libbeat, but this would be a big refactor at this point, and not related to this issue.

Any roadmap?

Take into account that multiple Beats can be combined as a single security solution, and this is not going to change.

Fair point

[jsoriano]

Yes, it can make sense to move all the common code to libbeat, but this would be a big refactor at this point, and not related to this issue.

Any roadmap?

There is no plan for a refactor like this at the moment afaik.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.