Feat/support aws sdk default credential provider chain

[aljoshare]

What this PR does

This PR allows to use the default credential provider chain of the AWS SDK. It is already part of the used thanos.io/objstore and can now also be enabled for Mimir. The default credential provider chain allows the use of e.g. role chaining using the ./aws/config or other authentication methods which are not implemented in thanos.io/objstore or Mimir.

Example config:

mimir:
  structuredConfig:
    common:
      storage:
        backend: s3
        s3:
          endpoint: s3.eu-central-1.amazonaws.com
          region: eu-central-1
          native_aws_auth_enabled: true
    blocks_storage:
      s3:
        bucket_name: mimir-bucket

Which issue(s) this PR fixes or relates to

Fixes #5613

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[CLAassistant]

All committers have signed the CLA.

[56quarters]

Thanks for this PR! I'm confused about why it's necessary though (and why it was added to thanos/objstore). It seems like Minio does indeed support the same authentication methods that the AWS SDK does:

https://github.com/minio/minio-go/tree/master/pkg/credentials

It seems like objstore configures a few of them by default

[aljoshare]

I was also confused and during my research I found a lot of confusion in many issues/PRs and discussions around the topic. My guess is that there is a lot of reinventing the wheel again instead of relying on the default provider chain of the SDK and of course legacy code which doesn't support all of the countless ways how to authenticate against AWS.

If I understand it correctly, the minio implementation had no way to use cross account role chaining when the aws_sdk_auth was introduced in the objstore [1][2]. It seems that it's supported...maybe, but I need to try it out to be really sure.

I can say that my use case (which was also the use case of the guy who introduced it in the objstore) is working properly now and I agree with him that relying on a consistent implementation from AWS is better than reimplementing it again and again and again.

What do you think?

[56quarters]

That makes sense to me. This is a much quicker way to let people use the auth logic they expect. My only suggestion is a different, slightly more descriptive name for the flag. What do you think about native_aws_auth_discovery_enabled?

[aljoshare]

native_aws_auth_enabled maybe? to keep it short?

Is it okay for you, if I fixup the existing commits? 😊

[56quarters]

native_aws_auth_enabled maybe? to keep it short?

Is it okay for you, if I fixup the existing commits? blush

Sure, sounds great. Thanks!

[56quarters]

I think CI is failing because of whitespace. Can you run make clean-white-noise and push again? I can merge after that. Thanks!

[56quarters]

LGTM, thanks!

[aljoshare]

make clean-white-noise didn't change anything but I recreated it with make reference-help and there were some whitespace changes. Looks good now, when running the tests locally 🤞