Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support pyarrow write_dataset with partitions #125

Merged
merged 3 commits into from
Aug 25, 2022
Merged

Support pyarrow write_dataset with partitions #125

merged 3 commits into from
Aug 25, 2022

Conversation

eddyxu
Copy link
Contributor

@eddyxu eddyxu commented Aug 24, 2022 •
edited
Loading

Closes #122

@eddyxu eddyxu requested a review from changhiskhan August 24, 2022 05:09
@eddyxu eddyxu self-assigned this Aug 24, 2022
@eddyxu eddyxu added python arrow Apache Arrow related issues labels Aug 24, 2022
@eddyxu eddyxu force-pushed the lei/py_write_dataset branch from 215854a to f99b7b5 Compare August 24, 2022 05:12
changhiskhan
changhiskhan reviewed Aug 24, 2022
View reviewed changes
Copy link
Contributor

@changhiskhan changhiskhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change examples / documentation to not use lance.write_table?

@eddyxu eddyxu force-pushed the lei/py_write_dataset branch from 4d6fb84 to 11b7d75 Compare August 24, 2022 23:34
@eddyxu
Copy link
Contributor Author

eddyxu commented Aug 24, 2022

We don't have an example on the Write path yet. Where do you think it is appropriate to add examples? Or can we address it later for docs altogether

changhiskhan
changhiskhan approved these changes Aug 25, 2022
View reviewed changes
Copy link
Contributor

@changhiskhan changhiskhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the calls to lance.write_table in bench_utils.py? And just skim the 3 README.md's to see if any of the examples need updating

@eddyxu
Copy link
Contributor Author

eddyxu commented Aug 25, 2022

Can you update the calls to lance.write_table in bench_utils.py? And just skim the 3 README.md's to see if any of the examples need updating

Sure, will do

@eddyxu eddyxu merged commit 9ffb998 into main Aug 25, 2022
@eddyxu eddyxu deleted the lei/py_write_dataset branch August 25, 2022 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@changhiskhan changhiskhan changhiskhan approved these changes

Assignees

@eddyxu eddyxu

Labels
arrow Apache Arrow related issues python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make Lance work with pyarrow.write_dataset API
2 participants
@eddyxu @changhiskhan