Skip to content
This repository has been archived by the owner on Dec 16, 2022. It is now read-only.

s3 unit tests failures #1666

Closed
hzeng-otterai opened this issue Aug 26, 2018 · 5 comments · Fixed by #1671
Closed

s3 unit tests failures #1666

hzeng-otterai opened this issue Aug 26, 2018 · 5 comments · Fixed by #1671

Comments

@hzeng-otterai
Copy link
Contributor

Describe the bug
I got unit tests failures on four s3 unit tests in allennlp/tests/common/file_utils_test.py on several my environments. The errors all happen at set_up_s3_bucket(). One of them is like:

$ pytest allennlp/tests/common/file_utils_test.py
=================================================================================== test session starts ===================================================================================
platform linux -- Python 3.6.6, pytest-3.7.2, py-1.5.4, pluggy-0.7.1
rootdir: /home/handsomezebra/dev/external/allennlp, inifile: pytest.ini
plugins: cov-2.5.1, flaky-3.4.0
collected 10 items                                                                                                                                                                        

allennlp/tests/common/file_utils_test.py ..FFFF....                                                                                                                                 [100%]

======================================================================================== FAILURES =========================================================================================
______________________________________________________________________________ TestFileUtils.test_s3_bucket _______________________________________________________________________________

self = <allennlp.tests.common.file_utils_test.TestFileUtils testMethod=test_s3_bucket>

    @mock_s3
    def test_s3_bucket(self):
        """This just ensures the bucket gets set up correctly."""
>       set_up_s3_bucket()

allennlp/tests/common/file_utils_test.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
allennlp/tests/common/file_utils_test.py:59: in set_up_s3_bucket
    s3_client.create_bucket(Bucket=bucket_name)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/client.py:314: in _api_call
    return self._make_api_call(operation_name, kwargs)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/client.py:599: in _make_api_call
    operation_model, request_dict)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/endpoint.py:102: in make_request
    return self._send_request(request_dict, operation_model)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/endpoint.py:131: in _send_request
    request = self.create_request(request_dict, operation_model)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/endpoint.py:115: in create_request
    operation_name=operation_model.name)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/hooks.py:227: in emit
    return self._emit(event_name, kwargs)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/hooks.py:360: in _emit
    aliased_event_name, kwargs, stop_on_response
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/hooks.py:210: in _emit
    response = handler(**kwargs)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/signers.py:90: in handler
    return self.sign(operation_name, request)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/signers.py:156: in sign
    auth.add_auth(request)
../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/auth.py:420: in add_auth
    super(S3SigV4Auth, self).add_auth(request)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <botocore.auth.S3SigV4Auth object at 0x7f23af88b940>, request = <botocore.awsrequest.AWSRequest object at 0x7f23af88b470>

    def add_auth(self, request):
        if self.credentials is None:
>           raise NoCredentialsError
E           botocore.exceptions.NoCredentialsError: Unable to locate credentials

../../../anaconda3/envs/py36/lib/python3.6/site-packages/botocore/auth.py:352: NoCredentialsError
------------------------------------------------------------------------------------ Captured log call ------------------------------------------------------------------------------------
16:49:27 - INFO - allennlp.common.checks - Pytorch version: 0.4.1
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable config_file from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable credentials_file from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable data_path from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable region from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable ca_bundle from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable api_versions from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable credentials_file from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable config_file from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable metadata_service_timeout from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable metadata_service_num_attempts from defaults.
16:49:27 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: env
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: assume-role
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: shared-credentials-file
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: custom-process
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: config-file
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: ec2-credentials-file
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: boto-config
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: container-role
16:49:27 - DEBUG - botocore.credentials - Looking for credentials via: iam-role
16:49:27 - DEBUG - urllib3.util.retry - Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
16:49:27 - DEBUG - botocore.utils - Caught exception while trying to retrieve credentials: Could not connect to the endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
Traceback (most recent call last):
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connection.py", line 171, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py", line 79, in create_connection
    raise err
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/util/connection.py", line 69, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/httpsession.py", line 242, in send
    decode_content=False,
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/util/retry.py", line 343, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/awsrequest.py", line 125, in _send_request
    method, url, body, headers, *args, **kwargs)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/awsrequest.py", line 152, in _send_output
    self.send(msg)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/awsrequest.py", line 236, in send
    return super(AWSConnection, self).send(str)
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connection.py", line 196, in connect
    conn = self._new_conn()
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/urllib3/connection.py", line 180, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPConnection object at 0x7f23afc2a588>: Failed to establish a new connection: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/utils.py", line 200, in _get_request
    response = self._session.send(request.prepare())
  File "/home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/httpsession.py", line 262, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"
16:49:27 - DEBUG - botocore.utils - Max number of attempts exceeded (1) when attempting to retrieve data from metadata service.
16:49:27 - DEBUG - botocore.loaders - Loading JSON file: /home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/data/endpoints.json
16:49:28 - DEBUG - botocore.session - Loading variable profile from defaults.
16:49:28 - DEBUG - botocore.hooks - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7f23ce4d9620>
16:49:28 - DEBUG - botocore.loaders - Loading JSON file: /home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/data/s3/2006-03-01/service-2.json
16:49:28 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x7f23ce51fea0>
16:49:28 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function lazy_call.<locals>._handler at 0x7f23afc6f730>
16:49:28 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x7f23ce51fc80>
16:49:28 - DEBUG - botocore.args - The s3 config key is not a dictionary type, ignoring its value of: None
16:49:28 - DEBUG - botocore.endpoint - Setting s3 timeout as (60, 60)
16:49:28 - DEBUG - botocore.loaders - Loading JSON file: /home/handsomezebra/anaconda3/envs/py36/lib/python3.6/site-packages/botocore/data/_retry.json
16:49:28 - DEBUG - botocore.client - Registering retry handlers for service: s3
16:49:28 - DEBUG - botocore.client - Defaulting to S3 virtual host style addressing with path style addressing fallback.
16:49:28 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <function validate_bucket_name at 0x7f23ce4ebd90>
16:49:28 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f23af8d3f60>>
16:49:28 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <function generate_idempotent_uuid at 0x7f23ce4eb9d8>
16:49:28 - DEBUG - botocore.hooks - Event before-call.s3.CreateBucket: calling handler <function add_expect_header at 0x7f23ce4ed2f0>
16:49:28 - DEBUG - botocore.hooks - Event before-call.s3.CreateBucket: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f23af8d3f60>>
16:49:28 - DEBUG - botocore.endpoint - Making request for OperationModel(name=CreateBucket) with params: {'url_path': '/my-bucket', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.8.1 Python/3.6.6 Linux/4.15.0-32-generic Botocore/1.11.1'}, 'body': b'', 'url': 'https://s3.amazonaws.com/my-bucket', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x7f23af9035c0>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'my-bucket'}}}
16:49:28 - DEBUG - botocore.hooks - Event request-created.s3.CreateBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f23af9035f8>>
16:49:28 - DEBUG - botocore.hooks - Event choose-signer.s3.CreateBucket: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f23afc2aac8>>
16:49:28 - DEBUG - botocore.hooks - Event choose-signer.s3.CreateBucket: calling handler <function set_operation_specific_signer at 0x7f23ce4eb8c8>
16:49:28 - DEBUG - botocore.hooks - Event before-sign.s3.CreateBucket: calling handler <function fix_s3_host at 0x7f23ce61a620>
16:49:28 - DEBUG - botocore.utils - Checking for DNS compatible bucket for: https://s3.amazonaws.com/my-bucket
16:49:28 - DEBUG - botocore.utils - URI updated to: https://my-bucket.s3.amazonaws.com/

To Reproduce
Steps to reproduce the behavior

  1. Using latest master branch
  2. pytest allennlp/tests/common/file_utils_test.py

System (please complete the following information):

  • OS: Linux Ubuntu
  • Python version: 3.6
  • AllenNLP version: master
  • PyTorch version: 0.4.1
@hzeng-otterai
Copy link
Contributor Author

hzeng-otterai commented Aug 26, 2018 •
edited
Loading

On my mac machine the error messages are different but still in the four s3 tests.

$ pytest allennlp/tests/common/file_utils_test.py 
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.6.6, pytest-3.6.2, py-1.5.3, pluggy-0.6.0
benchmark: 3.1.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/handsomezebra/dev/external/allennlp, inifile: pytest.ini
plugins: pythonpath-0.7.2, cov-2.5.1, benchmark-3.1.1, flaky-3.4.0
collected 10 items                                                                                                                                                                

allennlp/tests/common/file_utils_test.py ..FFFF....                                                                                                                         [100%]

==================================================================================== FAILURES =====================================================================================
__________________________________________________________________________ TestFileUtils.test_s3_bucket ___________________________________________________________________________

self = <allennlp.tests.common.file_utils_test.TestFileUtils testMethod=test_s3_bucket>

    @mock_s3
    def test_s3_bucket(self):
        """This just ensures the bucket gets set up correctly."""
>       set_up_s3_bucket()

allennlp/tests/common/file_utils_test.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
allennlp/tests/common/file_utils_test.py:59: in set_up_s3_bucket
    s3_client.create_bucket(Bucket=bucket_name)
../../../anaconda/envs/py36/lib/python3.6/site-packages/botocore/client.py:314: in _api_call
    return self._make_api_call(operation_name, kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <botocore.client.S3 object at 0x12112cf60>, operation_name = 'CreateBucket', api_params = {'Bucket': 'my-bucket'}

    def _make_api_call(self, operation_name, api_params):
        operation_model = self._service_model.operation_model(operation_name)
        service_name = self._service_model.service_name
        history_recorder.record('API_CALL', {
            'service': service_name,
            'operation': operation_name,
            'params': api_params,
        })
        if operation_model.deprecated:
            logger.debug('Warning: %s.%s() is deprecated',
                         service_name, operation_name)
        request_context = {
            'client_region': self.meta.region_name,
            'client_config': self.meta.config,
            'has_streaming_input': operation_model.has_streaming_input,
            'auth_type': operation_model.auth_type,
        }
        request_dict = self._convert_to_request_dict(
            api_params, operation_model, context=request_context)
    
        handler, event_response = self.meta.events.emit_until_response(
            'before-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            model=operation_model, params=request_dict,
            request_signer=self._request_signer, context=request_context)
    
        if event_response is not None:
            http, parsed_response = event_response
        else:
            http, parsed_response = self._endpoint.make_request(
                operation_model, request_dict)
    
        self.meta.events.emit(
            'after-call.{endpoint_prefix}.{operation_name}'.format(
                endpoint_prefix=self._service_model.endpoint_prefix,
                operation_name=operation_name),
            http_response=http, parsed=parsed_response,
            model=operation_model, context=request_context
        )
    
        if http.status_code >= 300:
            error_code = parsed_response.get("Error", {}).get("Code")
            error_class = self.exceptions.from_code(error_code)
>           raise error_class(parsed_response, operation_name)
E           botocore.errorfactory.BucketAlreadyExists: An error occurred (BucketAlreadyExists) when calling the CreateBucket operation: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.

../../../anaconda/envs/py36/lib/python3.6/site-packages/botocore/client.py:612: BucketAlreadyExists
-------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------
16:31:20 - INFO - allennlp.common.checks - Pytorch version: 0.4.1
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable config_file from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable credentials_file from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable data_path from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable region from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable ca_bundle from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable api_versions from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable credentials_file from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable config_file from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable metadata_service_timeout from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable metadata_service_num_attempts from defaults.
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.credentials - Looking for credentials via: env
16:31:20 - DEBUG - botocore.credentials - Looking for credentials via: assume-role
16:31:20 - DEBUG - botocore.credentials - Looking for credentials via: shared-credentials-file
16:31:20 - INFO - botocore.credentials - Found credentials in shared credentials file: ~/.aws/credentials
16:31:20 - DEBUG - botocore.loaders - Loading JSON file: /Users/handsomezebra/anaconda/envs/py36/lib/python3.6/site-packages/botocore/data/endpoints.json
16:31:20 - DEBUG - botocore.session - Loading variable profile from defaults.
16:31:20 - DEBUG - botocore.hooks - Event choose-service-name: calling handler <function handle_service_name_alias at 0x118f36598>
16:31:20 - DEBUG - botocore.loaders - Loading JSON file: /Users/handsomezebra/anaconda/envs/py36/lib/python3.6/site-packages/botocore/data/s3/2006-03-01/service-2.json
16:31:20 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x118ef6ea0>
16:31:20 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function lazy_call.<locals>._handler at 0x120a7ef28>
16:31:20 - DEBUG - botocore.hooks - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x118ef6c80>
16:31:20 - DEBUG - botocore.args - The s3 config key is not a dictionary type, ignoring its value of: None
16:31:20 - DEBUG - botocore.endpoint - Setting s3 timeout as (60, 60)
16:31:20 - DEBUG - botocore.loaders - Loading JSON file: /Users/handsomezebra/anaconda/envs/py36/lib/python3.6/site-packages/botocore/data/_retry.json
16:31:20 - DEBUG - botocore.client - Registering retry handlers for service: s3
16:31:20 - DEBUG - botocore.client - Defaulting to S3 virtual host style addressing with path style addressing fallback.
16:31:20 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <function validate_bucket_name at 0x118f4cc80>
16:31:20 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x1211da160>>
16:31:20 - DEBUG - botocore.hooks - Event before-parameter-build.s3.CreateBucket: calling handler <function generate_idempotent_uuid at 0x118f4c8c8>
16:31:20 - DEBUG - botocore.hooks - Event before-call.s3.CreateBucket: calling handler <function add_expect_header at 0x118f4e1e0>
16:31:20 - DEBUG - botocore.hooks - Event before-call.s3.CreateBucket: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x1211da160>>
16:31:20 - DEBUG - botocore.endpoint - Making request for OperationModel(name=CreateBucket) with params: {'url_path': '/my-bucket', 'query_string': {}, 'method': 'PUT', 'headers': {'User-Agent': 'Boto3/1.8.0 Python/3.6.6 Darwin/17.7.0 Botocore/1.11.1'}, 'body': b'', 'url': 'https://s3.amazonaws.com/my-bucket', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0x12119a6a0>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'my-bucket'}}}
16:31:20 - DEBUG - botocore.hooks - Event request-created.s3.CreateBucket: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x121138358>>
16:31:20 - DEBUG - botocore.hooks - Event choose-signer.s3.CreateBucket: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x120ef9518>>
16:31:20 - DEBUG - botocore.hooks - Event choose-signer.s3.CreateBucket: calling handler <function set_operation_specific_signer at 0x118f4c7b8>
16:31:20 - DEBUG - botocore.hooks - Event before-sign.s3.CreateBucket: calling handler <function fix_s3_host at 0x118dfd620>
16:31:20 - DEBUG - botocore.utils - Checking for DNS compatible bucket for: https://s3.amazonaws.com/my-bucket
16:31:20 - DEBUG - botocore.utils - URI updated to: https://my-bucket.s3.amazonaws.com/
16:31:20 - DEBUG - botocore.auth - Calculating signature using v4 auth.
16:31:20 - DEBUG - botocore.auth - CanonicalRequest:
PUT
/

host:my-bucket.s3.amazonaws.com
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180825T233120Z

host;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
16:31:20 - DEBUG - botocore.auth - StringToSign:
AWS4-HMAC-SHA256
20180825T233120Z
20180825/us-east-1/s3/aws4_request
9e5dfe3d11e3bd48a67bf5d3d0274d9a0a5b06ddf8581a8ae420801d9cac1f6c
16:31:20 - DEBUG - botocore.auth - Signature:
d32056b7f6088f562685599105c6a935c2e89edbb86d64036cbb1c4befa6d663
16:31:20 - DEBUG - botocore.endpoint - Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=https://my-bucket.s3.amazonaws.com/, headers={'User-Agent': b'Boto3/1.8.0 Python/3.6.6 Darwin/17.7.0 Botocore/1.11.1', 'X-Amz-Date': b'20180825T233120Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=AKIAI2QSHPI2VBKJAWOA/20180825/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=d32056b7f6088f562685599105c6a935c2e89edbb86d64036cbb1c4befa6d663', 'Content-Length': '0'}>
16:31:20 - DEBUG - urllib3.util.retry - Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None)
16:31:20 - DEBUG - botocore.parsers - Response headers: {'x-amz-bucket-region': 'us-east-1', 'x-amz-request-id': '9BF74403CAF136A2', 'x-amz-id-2': 'h16Ho9ai0KeoeqJp6hzMHlFAhwcIlcwEoLjZRaGFXy1YOEac1A0CRvzZ7Bwe04U58BxwLaKPSGg=', 'Content-Type': 'application/xml', 'Transfer-Encoding': 'chunked', 'Date': 'Sat, 25 Aug 2018 23:31:20 GMT', 'Server': 'AmazonS3'}
16:31:20 - DEBUG - botocore.parsers - Response body:
b'<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>BucketAlreadyExists</Code><Message>The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.</Message><BucketName>my-bucket</BucketName><RequestId>9BF74403CAF136A2</RequestId><HostId>h16Ho9ai0KeoeqJp6hzMHlFAhwcIlcwEoLjZRaGFXy1YOEac1A0CRvzZ7Bwe04U58BxwLaKPSGg=</HostId></Error>'
16:31:20 - DEBUG - botocore.hooks - Event needs-retry.s3.CreateBucket: calling handler <botocore.retryhandler.RetryHandler object at 0x12119add8>
16:31:20 - DEBUG - botocore.retryhandler - No retry needed.
16:31:20 - DEBUG - botocore.hooks - Event needs-retry.s3.CreateBucket: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x1211da160>>

@joelgrus
Copy link
Contributor

it looks like the issue is this one:

getmoto/moto#1793

I was able to break that the file_utils_test for myself by doing

pip install --upgrade botocore

and fix it by doing

pip install botocore==1.10.38

other people have fixed it by pinning boto3:

https://github.com/ImmobilienScout24/ultimate-source-of-aws-accounts/pull/2/files#diff-2955d5257f635de1df53f55a171ca5c7R32

not sure what the best fix is

@matt-gardner
Copy link
Contributor

Since we depend on boto3, not botocore, seems like we should just pin boto3?

@joelgrus
Copy link
Contributor

that works, I'll make a PR

@joelgrus joelgrus mentioned this issue Aug 27, 2018
Merged
@joelgrus
Copy link
Contributor

ugh, this is not so simple because awscli requires botocore==1.11.1

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

pin boto3 joelgrus/allennlp
3 participants
@joelgrus @matt-gardner @hzeng-otterai