Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parallel reads seems to be turned off in lance? #93

Closed
Tracked by #97
changhiskhan opened this issue Aug 11, 2022 · 2 comments
Closed
Tracked by #97

parallel reads seems to be turned off in lance? #93

changhiskhan opened this issue Aug 11, 2022 · 2 comments
Labels
c++ C++ issues duckdb enhancement New feature or request

Comments

@changhiskhan
Copy link
Contributor

I ran the label distribution query against coco for parquet and lance. For parquet, the total CPU time was greater than the Wall time, but for lance that was not the case. I vaguely remember discussing this before, not sure if it's already a known issue.
@changhiskhan changhiskhan mentioned this issue Aug 11, 2022
Merged
5 tasks
@eddyxu
Copy link
Contributor

eddyxu commented Aug 13, 2022

Yes, currently we disabled parallel reads when LIMIT / OFFSET is provided for simpler design for Limit exector. We can revise this decision though.

@eddyxu eddyxu added enhancement New feature or request c++ C++ issues duckdb labels Aug 13, 2022
@changhiskhan
Copy link
Contributor Author

there's no limit/offset in the label distribution query tho. Is this because of the batch size thing?

@eddyxu eddyxu closed this as completed Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c++ C++ issues duckdb enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eddyxu @changhiskhan