Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose AWS creds and region configurability to python #1155

Closed
chebbyChefNEQ opened this issue Aug 21, 2023 · 0 comments · Fixed by #2147
Closed

Expose AWS creds and region configurability to python #1155

chebbyChefNEQ opened this issue Aug 21, 2023 · 0 comments · Fixed by #2147
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@chebbyChefNEQ
Copy link
Contributor

We added aws creds injection in #1059 and region injection in #1153. Currently these APIs are not exposed to python binding, where user still have to rely on the default cred chain to get creds. This is okay most of the time, but it would be nice to be able to use custom creds for situations like assume role.
@chebbyChefNEQ chebbyChefNEQ added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Aug 21, 2023
@wjones127 wjones127 mentioned this issue Apr 4, 2024
Merged
wjones127 added a commit that referenced this issue Apr 5, 2024
@wjones127
feat: easier and consistent S3 configuration (#2147)
3c9234b 
This PR makes several changes:

1. Fixes an issue where `StorageOptions` would be bypassed in favor of
`DefaultCredentialsChain` for the commit lock.
2. Makes credentials in `storage_options` take precedence over
`DefaultCredentialsChain`. This way credentials passed in through
function parameters will actually be used. Closes #1155
3. Fixes an issue were `Dataset.checkout_version()` would cause the
dataset to lose it's storage configurations.
4. Refactored the integration tests so they are easier to run locally.
`pytest` now handles the setup and teardown of buckets and dynamodb
tables. Users just have to run `docker compose up`. Also, this is now
documented in the `DEVELOPMENT.md`.
5. Bucket region is now automatically detected when using AWS S3.
6. We no longer hardcode allowing insecure connections to S3
(`allow_http(true)`). Users can set this themselves if they want it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: easier and consistent S3 configuration wjones127/lance
1 participant
@chebbyChefNEQ