-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
s3 bucket existence - unexpected behavior #796
Comments
Thank you for reporting the issue. The It looks like Amazon S3 returns 301 permanent redirects for existing buckets when you head it in the wrong region. The fix would be to treat this status code as a successful check. Yes the bucket exists. |
The fix was very straight forward. I've added a test to cover regressions. This should go out later this week. Thank you for submitting the bug report and for your patience! |
[IDSEQ-1643] Fix bug for cross-region S3 uploads### Description - This is just to address the cross-region copying issue from last week. See JIRA for details. - Using the S3 "global" non-regional specific endpoint means that AWS will resolve the region itself. ### Notes - This was the most elegant solution I could find. My original proposal was to use `GetBucketLocation` but apparently it is only for the bucket owner (https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html). I also found suggestions to use head bucket but got 301 redirects / couldn't figure out how to get the region info from that (aws/aws-sdk-ruby#796). - This was the only use of S3_CLIENT that I found necessary to change, and it was the only one changed in the previous PR (#2538). - Why not set this global endpoint all the time? AWS recommends against it "Please configure the proper region to avoid multiple unnecessary redirects and signing attempts". And for most other requests we're in our own buckets in our known region. ### Tests - Testing with the reported user URL. #### Before: ![Screen Shot 2019-10-29 at 2 59 37 PM](https://user-images.githubusercontent.com/5652739/67895204-4a3f6900-fb17-11e9-8a60-c4b9b05d36b6.png) #### After: ![Screen Shot 2019-10-29 at 3 02 12 PM](https://user-images.githubusercontent.com/5652739/67895214-4f9cb380-fb17-11e9-81b9-698bc63ff13d.png) #### Ran samples end-to-end locally and it works: <img width="1088" alt="Screen Shot 2019-10-30 at 12 53 59 PM" src="https://user-images.githubusercontent.com/5652739/67895228-54f9fe00-fb17-11e9-9d5b-69c19cbcf898.png">
The code below demonstrates unexpected behavior when working with buckets in version 2.0.4.1. See also #460 (comment).
The text was updated successfully, but these errors were encountered: