A way to add custom HTTP headers to S3 calls

[arainchik]

Is your feature request related to a problem? Please describe.
We need a way to add custom HTTP headers to S3 calls without using global variables

Describe the solution you'd like
Current implementation allows you to add custom HTTP headers using event system as described in #2251 but it does not allow you to pass headers from the application making such calls. The only possible way I see right now is to use global variables, which is not a good practice.

My suggestion is to add new parameter CustomHTTPHeaders for S3 calls in boto, which will be passed to S3 target.

An example would be:

import boto3
s3 = boto3.client('s3')

s3.put_object(Bucket='BUCKET', Body=b'foo bar', CustomHTTPHeaders={'foo': 'bar'} )
print('Done')

[stobrien89]

Hi @arainchik,

Thanks for the feature request! Would you mind elaborating on what your particular use case would be for this?

[arainchik]

There are multiple uses for this kind of feature. One example would be to pass special authentication token in HTTP headers to HTTPS proxy.

[stobrien89]

Hi @arainchik,

Thanks for clarifying! I'll pass this along to the team.

[andyalbert]

Hi @stobrien89
Can I pick this one up?

[stobrien89]

Hi @andyalbert,

My apologies for the delay. Let me double-check with the team that we'll accept a change like this before you put in the effort. I'll probably have an update sometime next week.

[stobrien89]

Hi all,

Just to confirm, the main issue here is that you're unable to dynamically pass headers to the event system, correct?

[arainchik]

@stobrien89 you are correct, there is no way to pass custom data to event handlers, currently we have to do it in a non-pythonic way using global variables/queues.

[bradhill99]

Found possible solution from https://stackoverflow.com/questions/58828800/adding-custom-headers-to-all-boto3-requests

def _add_header(request, **kwargs):
    request.headers.add_header('x-trace-id', 'trace-trace')
    print(request.headers)  # for debug

some_client = boto3.client(service_name=SERVICE_NAME)
event_system = some_client.meta.events
event_system.register_first('before-sign.*.*', _add_header)

[arainchik]

Found possible solution from https://stackoverflow.com/questions/58828800/adding-custom-headers-to-all-boto3-requests

def _add_header(request, **kwargs):
    request.headers.add_header('x-trace-id', 'trace-trace')
    print(request.headers)  # for debug

some_client = boto3.client(service_name=SERVICE_NAME)
event_system = some_client.meta.events
event_system.register_first('before-sign.*.*', _add_header)

It works, but as I mintioned earlier:

there is no way to pass custom data to event handlers, currently we have to do it in a non-pythonic way using global variables/queues.

[kdaily]

Hi all, closing this in favor of a duplicate opened at botocore:

boto/botocore#2050

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.