Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parquet AsyncArrowWriter not shutting down inner async writer. #4058

Closed
kindly opened this issue Apr 12, 2023 · 1 comment · Fixed by #4059
Closed

Parquet AsyncArrowWriter not shutting down inner async writer. #4058

kindly opened this issue Apr 12, 2023 · 1 comment · Fixed by #4059
Labels
bug parquet Changes to the parquet crate

Comments

@kindly
Copy link
Contributor

kindly commented Apr 12, 2023

Describe the bug

I have tried to use the new parquet AsyncArrowWriter by writing files to S3 using the object_store multipart upload. However, for the upload to complete, the inner writer needs to be shutdown and this writer does not do that.

Additional context

Pull request to shut down on close will be in pull request.
@kindly kindly added the bug label Apr 12, 2023
@kindly kindly mentioned this issue Apr 12, 2023
Merged
@tustvold tustvold added the parquet Changes to the parquet crate label Apr 21, 2023
@tustvold
Copy link
Contributor

label_issue.py automatically added labels {'parquet'} from #4059

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Shutdown parquet async writer kindly/arrow-rs
2 participants
@kindly @tustvold