Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve partition pruning during file listing #205

Open
xushiyan opened this issue Nov 30, 2024 · 3 comments
Open

Improve partition pruning during file listing #205

xushiyan opened this issue Nov 30, 2024 · 3 comments
Assignees
@TheR1sing3un
Labels
p1 refactor Code refactoring without any functionality or behavior change
Milestone
release-0.3.0

Comments

@xushiyan
Copy link
Member

During partition loading (

hudi-rs/crates/core/src/table/fs_view.rs

Line 60 in d90abed

async fn load_partition_paths(
) , the file listing can be shortcut by partition pruner to reduce potentially a lot of file listing calls.
@xushiyan xushiyan added refactor Code refactoring without any functionality or behavior change p1 labels Nov 30, 2024
@xushiyan xushiyan added this to the release-0.3.0 milestone Nov 30, 2024
@xushiyan
Copy link
Member Author

@TheR1sing3un are you interested in picking this up?

@TheR1sing3un
Copy link
Member

@TheR1sing3un are you interested in picking this up?

Yes, I'm willing to try！Thanks u!

@xushiyan
Copy link
Member Author

hi @TheR1sing3un , any progress on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TheR1sing3un TheR1sing3un

Labels
p1 refactor Code refactoring without any functionality or behavior change
Projects
None yet
Milestone
release-0.3.0
Development

No branches or pull requests
2 participants
@xushiyan @TheR1sing3un