Add support for EKS Pod Identity

[ripa1993]

Is your feature request related to a problem? Please describe.

AWS has introduced EKS Pod Identity as a new way for authenticating Kubernetes Pods against AWS APIs.

Describe the solution you'd like

This requires upgrading the Go AWS SDK v2 to at least 1.23.0, currently 1.16.0 is being used by dependency thanos-io/objstore

Describe alternatives you've considered

Using IRSA (iam roles for service accounts) for Mimir, but I would prefer to stick with EKS Pod Identity for uniformity with other tools running in Kubernetes

Additional context

Add any other context or screenshots about the feature request here.

Linked issue: thanos-io/objstore#131

[armandgrillet]

Thank you for the issue. As stated, the bump first needs to happen in thanos-io/objstore. Will there be any benefits doing that change?

[skpaz]

Thanos appears to support EKS Pod Identities as of this PR: thanos-io/thanos#7335

They mentioned it would be available in v0.36.0 here: thanos-io/thanos#7156 (comment)

An individual noted that it required aws_sdk_auth=true here: thanos-io/objstore#131

[francoposa]

@skpaz The comment on thanos-io/objstore#131 says that it is working in the current version (at least for that user) but aws_sdk_auth must be false.

In Mimir, it defaults to false, and can be set by the native_aws_auth_enabled config -common.storage.s3.native-aws-auth-enabled.

Loki is also just just using the thanos-io/objstore on a recent version but only with github.com/aws/aws-sdk-go-v2 v1.16.0 dependency .
Do we have information on what it looks like when it does work in Loki but does not work in Mimir?

[skpaz]

@francoposa I have a GEL cluster up w/ Pod Identities in AWS and a complete write-up of the steps taken, Helm values.yaml, etc.

Want to touch base next week? I'm not sure what you need, but if I can provide it, I'd be happy to help.