[libbeat] fix: aws & openstack metadata conflict in add_cloud_metadata processor

[Kavindu-Dodan]

Proposed commit message

This PR fixes incorrect recognition of EC2/AWS cloud provider as Openstack. The root cause was the common metadata endpoints used by both AWS SDK & Openstack logic. And this happened when IMDSv2 is disabled in AWS.

I attempted to migrate Openstack logic to another metadata implementation. However, I did not manage to create a fully functioning setup to validate the implementation. Hence, this PR focuses on a priority-based solution where priority is given for SDK-backed metadata fetching over HTTP endpoints.

Current priory providers are - aws/ec2 & azure

Note - I have done a minor refactoring to rename Local struct property to DefaultEnabled to make intention clearer

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

You need a local copy and an EC2 instance to validate the fix.

• Enable metadata service in EC2 instance and make IMDSv2 optional
• Build a beats (ex:- metricbeat) module based on this libbeat change
• Copy the beats module to EC2 instance and start the module with add_cloud_metadata processor enabled & logs set to debug for more in-depth logs & no provider enforced
• Observe logs and see data (ex:- system monitoring) through Kibana to validate cloud provider detection

Related issues

• Closes add_cloud_metadata detecting wrong cloud provider (aws as openstack) #13816

Screenshots

-IMDSv2 disabled

• Processor enabled but no provider enforced

• Debug logs on multi-result and priority based selection

• Cloud provider detected correctly,

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[kaiyan-sheng]

Thanks for working on this issue! Can we document this new logic around priorityProviders in the doc please?

[Kavindu-Dodan]

Thanks for working on this issue! Can we document this new logic around priorityProviders in the doc please?

Thank you, I added documentation :)

[axw]

@Kavindu-Dodan with this change could we end up having the opposite problem, where OpenStack is misidentified as AWS?

[Kavindu-Dodan]

@Kavindu-Dodan with this change could we end up having the opposite problem, where OpenStack is misidentified as AWS?

This shouldn't happen as priority is given for cloud provider SDK based metadata fetchers (AWS & Azure ¹), where they should fail if configurations are not set (ex:- AWS credential profile/env vars).

Footnotes

1. https://github.com/elastic/beats/pull/41636/files#diff-54e31882d706ede511023836a842dd7c1e570ce1eab2c7208601afacee49a995R76-R79 ↩