Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Compression in parquet-fromcsv #4160

Merged
merged 6 commits into from
May 3, 2023
Merged

Support Compression in parquet-fromcsv #4160

merged 6 commits into from
May 3, 2023

Conversation

suxiaogang223
Copy link
Contributor

Which issue does this PR close?

Closes #3721

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

add new arg for parquet-fromcsv : -C/--csv-commoression

@github-actions github-actions bot added the parquet Changes to the parquet crate label Apr 30, 2023
@tustvold tustvold changed the title feat: Support compressed Support Compression in parquet-fromcsv May 2, 2023
tustvold
tustvold reviewed May 2, 2023
View reviewed changes
Copy link
Contributor

@tustvold tustvold left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good thank you, only thing left I think is some feature flag shenanigans

I believe running with cargo run --bin parquet-from-csv --no-default-features --features arrow,cli will result in compilation errors now

parquet/src/bin/parquet-fromcsv.rs Outdated Show resolved Hide resolved
parquet/src/bin/parquet-fromcsv.rs Outdated Show resolved Hide resolved
parquet/src/bin/parquet-fromcsv.rs Outdated Show resolved Hide resolved
parquet/src/bin/parquet-fromcsv.rs Outdated Show resolved Hide resolved
@suxiaogang223
Copy link
Contributor Author

Looking good thank you, only thing left I think is some feature flag shenanigans

I believe running with cargo run --bin parquet-from-csv --no-default-features --features arrow,cli will result in compilation errors now

thanks, I'll fix this soon

@tustvold tustvold merged commit 547cb80 into apache:master May 3, 2023
@suxiaogang223 suxiaogang223 deleted the support_compressed branch May 4, 2023 00:35
@tustvold tustvold mentioned this pull request May 18, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tustvold tustvold tustvold approved these changes

Assignees
No one assigned
Labels
parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support compressed CSV/TSV files in parquet-fromcsv.
2 participants
@suxiaogang223 @tustvold