Add IMDS fallback with DescribeInstances.

[jefchien]

Description: Similar to aws/amazon-cloudwatch-agent#1139

Adds a fallback metadata provider that calls EC2 DescribeInstances with a filter based on the hostname.

Consolidates the metadata provider functions into a single package (internal/metadataprovider). New usages of the package

• In internal/aws/awsutil/conn.go to determine EC2 region.
• In receiver/awscontainerinsightsreceiver/internal/host/ec2metadata.go.
• In internal/aws/xray/telemetry/sender.go to get instance ID and hostname.

Link to tracking Issue: aws/amazon-cloudwatch-agent#1101

Testing: Updated unit tests. Built the agent image and used on EKS cluster with a hop limit of 1.

Without permissions for ec2:DescribeInstances:

D! [EC2] Found active network interface
I! imds retry client will retry 1 times
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
E! [EC2] Fetch identity document from EC2 metadata fail: unable to get metadata from IMDSv2: EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: EC2MetadataError: failed to get IMDSv2 token and fallback to IMDSv1 is disabled
caused by: : 
    status code: 0, request id: 
caused by: RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
unable to get metadata from IMDSv1: EC2MetadataRequestError: failed to get EC2 instance identity document
caused by: EC2MetadataError: failed to make EC2Metadata request

    status code: 401, request id: 
unable to get metadata from DescribeInstances: UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::012345678901:assumed-role/test/eks-test-cloudwatch-a12ef9b5-47d9-4deb-a0ed-1e3b4b757929 is not authorized to perform: ec2:DescribeInstances because no identity-based policy allows the ec2:DescribeInstances action
    status code: 403, request id: 8358ddcd-a28c-444b-b7d6-efcd2767ff9b
2024/04/18 14:31:42 I! attempt to access ECS task metadata to determine whether I'm running in ECS.
2024/04/18 14:31:43 W! retry [0/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2024/04/18 14:31:44 W! retry [1/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
.
.
.
2024-04-18T14:31:45Z E! {"caller":"[email protected]/conn.go:388","msg":"Failed to get credential from session","kind":"exporter","data_type":"metrics","name":"awsemf/containerinsights","error":"SharedCredsLoad: failed to load shared credentials file\ncaused by: FailedRead: unable to open file\ncaused by: open /root/.aws/credentials: no such file or directory","stacktrace":"github.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil.GetDefaultSession\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/[email protected]/conn.go:388\ngithub.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil.(*Conn).newAWSSession\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/[email protected]/conn.go:269\ngithub.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/awsutil.GetAWSConfigSession\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/internal/aws/[email protected]/conn.go:216\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/awsemfexporter.newEmfExporter\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/[email protected]/emf_exporter.go:61\ngithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/awsemfexporter.createMetricsExporter\n\tgithub.com/open-telemetry/opentelemetry-collector-contrib/exporter/[email protected]/factory.go:58\ngo.opentelemetry.io/collector/exporter.CreateMetricsFunc.CreateMetricsExporter\n\tgo.opentelemetry.io/collector/[email protected]/exporter.go:113\ngo.opentelemetry.io/collector/exporter.(*Builder).CreateMetrics\n\tgo.opentelemetry.io/collector/[email protected]/exporter.go:245\ngo.opentelemetry.io/collector/service/internal/graph.(*exporterNode).buildComponent\n\tgo.opentelemetry.io/collector/[email protected]/internal/graph/nodes.go:191\ngo.opentelemetry.io/collector/service/internal/graph.(*Graph).buildComponents\n\tgo.opentelemetry.io/collector/[email protected]/internal/graph/graph.go:287\ngo.opentelemetry.io/collector/service/internal/graph.Build\n\tgo.opentelemetry.io/collector/[email protected]/internal/graph/graph.go:73\ngo.opentelemetry.io/collector/service.(*Service).initExtensionsAndPipeline\n\tgo.opentelemetry.io/collector/[email protected]/service.go:225\ngo.opentelemetry.io/collector/service.New\n\tgo.opentelemetry.io/collector/[email protected]/service.go:129\ngo.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents\n\tgo.opentelemetry.io/collector/[email protected]/collector.go:171\ngo.opentelemetry.io/collector/otelcol.(*Collector).Run\n\tgo.opentelemetry.io/collector/[email protected]/collector.go:229\ngo.opentelemetry.io/collector/otelcol.NewCommand.func1\n\tgo.opentelemetry.io/collector/[email protected]/command.go:27\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:983\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:1115\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:1039\nmain.runAgent\n\tgithub.com/aws/amazon-cloudwatch-agent/cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go:356\nmain.reloadLoop\n\tgithub.com/aws/amazon-cloudwatch-agent/cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go:177\nmain.main\n\tgithub.com/aws/amazon-cloudwatch-agent/cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go:603\nruntime.main\n\truntime/proc.go:250"}

With ec2:DescribeInstances permissions:

D! [EC2] Found active network interface
I! imds retry client will retry 1 times
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
D! should retry true for imds error : RequestError: send request failed
caused by: Put "http://169.254.169.254/latest/api/token": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I! Detected the instance is EC2
.
.
.
2024-04-18T14:41:07Z I! {"caller":"internal/resourcedetection.go:139","msg":"detected resource information","kind":"processor","name":"resourcedetection","pipeline":"metrics/application_signals","resource":{"cloud.account.id":"012345678901","cloud.availability_zone":"us-west-1c","cloud.platform":"aws_eks","cloud.provider":"aws","cloud.region":"us-west-1","ec2.tag.aws:autoscaling:groupName":"eks-ng-41116254-70c70db9-6c24-3115-bdf0-08c29aa7bd84","ec2.tag.kubernetes.io/cluster/test":"owned","host.id":"i-0123456789abcdef","host.image.id":"ami-0123456789abcdef0","host.name":"ip-10-24-34-0.us-west-1.compute.internal","host.type":"m5.large"}}
.
.
.
2024-04-18T14:41:13Z I! {"caller":"host/ec2tags.go:131","msg":"Fetch ec2 tags successfully","kind":"receiver","name":"awscontainerinsightreceiver","data_type":"metrics"}

Can get EC2 instance metadata dimensions for container insights and app signals.

Documentation: N/A