s3.doesBucketExist from us-east-1 on a bucket in us-west-2 fails with 400: Bad Request

[andremoeller]

Hello,

Similar to aws/aws-sdk-ruby#1267:

I have a bucket 'foo' that exists in us-west-2. Using an s3 client configured for us-east-1, calling s3Client.doesBucketExist("foo") gives me

User class threw exception: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request

(I'd expect it to just return "true")

Calling createBucket or getBucketAcl (which doesBucketExistV2 does) with the same bucket name gives me

model.AmazonS3Exception: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'us-west-2' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed

I'm using aws sdk java 1.11. Using a us-west-2 s3 client works as I'd expect.

Thanks!

[varunnvs92]

Java SDK by default uses SigV4 signing. For S3, bucket region should be included during SigV4 signing. So When you create a client with "us-east-1" region and make any call, the call will be signed with "us-east-1" region. But as the bucket you are calling is in "us-west-2" the call fails with malformed authorization header. We suggest creating regional clients and interact only with buckets in that region. For example, if you want to talk to us-west-2 region, create a s3 client with "us-west-2" region.

Let us know if you have more questions. Thank you.

[andremoeller]

Understood, thanks. I have a couple of questions:

Why does this not occur on a bucket in us-east-1 if my client is configured with us-west-2? If I call doesBucketExist, I get true (expected). If I try to getBucketAcl, I get a 301 with "The bucket is in this region: us-east-1." (also expected). From your comment, I'd expect these calls to fail due to a malformed authorization header since they're signed with "us-west-2"

If I attempt to create a bucket that already exists in us-west-2 with a client in us-east-1, I get an exception with error code AuthorizationHeaderMalformed, not BucketAlreadyExists or BucketAlreadyOwnedByYou (or a 200 OK, which I'd expect from http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html). However, if I attempt to create a bucket that already exists in us-east-1 with a client in us-west-2, I get BucketAlreadyOwnedByYou, as expected -- what accounts for this discrepancy in behavior?

Thanks!

[varunnvs92]

This is a little tricky concept. S3's us-east-1 region is not exactly a region endpoint. It is a global endpoint. That is when you create a client with us-east-1, your request will go to "s3.amazonaws.com" which will do DNS magic and redirect the request to the correct bucket endpoint. It means you can use "us-east-1" to interact with buckets in "us-east-1" and other regions but you have to sign the request with proper bucket location. This is the reason you see different behavior with us-east-1 client compared to clients in other regions.

There are many edge cases when you try to make cross region calls which is why we don't suggest using single S3 client to talk to different regions. S3 was one of the initial services AWS supported and so there are inconsistencies which we can't change due to backward compatible reasons.
I can't exactly answer the specific behaviors you mentioned as it depends on the resolution logic (endpoint resolution, auth validation etc) S3 implements in their service. Thank you.

[varunnvs92]

I can try to answer the second case. This is my understanding but may not be exact as I am not from S3 team.

Lets start with easy case:
Create a bucket that already exists in us-east-1 - with a client in us-west-2
The request is signed with us-west-2 (set on client) and send to s3.us-west-2.amazonaws.com endpoint. As signing region and endpoint are same, authorization succeeds. Now S3 checks if bucket exists before creating it and finds that a bucket already exists in us-east-1. So you get BucketAlreadyOwnedByYou message.

Create a bucket that already exists in us-west-2 with a client in us-east-1
The request is signed with "us-east-1" and send to s3.amazonaws.com endpoint. As I mentioned this is a special endpoint which somehow finds that your bucket is in us-west-2 and redirects the request to that endpoint. This means you are sending a request to us-west-2 endpoint but signing it with "us-east-1", hence you get the AuthorizationHeaderMalformed. Authorization is done before any API specific logic.

[BalabhadraVenkat]

When I used 'https://us-east-1.s3.amazonaws.com' and upload the content to 'bucket', it used to fail with saying bucket not found. But if I create a dummy bucket 'bucket.us-east-1' it used to work.

After a while, realized that using the endpoint 's3.amazonaws.com' without any region in it, fixed the error.

[jsumit5]

@BalabhadraVenkat i am also facing the same issue. as suggested i removed us-west-2.

    def s3 = AmazonS3ClientBuilder.standard()
    .withEndpointConfiguration(new EndpointConfiguration("https://s3.amazonaws.com",file_region))
    .withCredentials(new AWSStaticCredentialsProvider(basicSessionCredentials))
    .withPathStyleAccessEnabled(true)
    .build() 

but now i am getting below error.

com.amazonaws.services.s3.model.AmazonS3Exception: The bucket is in this region: us-west-2

Can you tell me what change you have done?