Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(core/services/hdfs): Fix the HDFS write failure when atomic_write_dir is set #5039

Merged
merged 7 commits into from
Sep 3, 2024
Merged

fix(core/services/hdfs): Fix the HDFS write failure when atomic_write_dir is set #5039

merged 7 commits into from
Sep 3, 2024

Conversation

meteorgan
Copy link
Contributor

@meteorgan meteorgan commented Aug 23, 2024 •
edited
Loading

Which issue does this PR close?

Closes #4867.

Rationale for this change

What changes are included in this PR?

  1. add hdfs cluster to test atomic_write_dir
  2. migrate service_test_hdfs to test planner
  3. fix write failure

Are there any user-facing changes?

@meteorgan meteorgan marked this pull request as ready for review August 23, 2024 15:59
@meteorgan
Copy link
Contributor Author

There's a lot of duplicate code between the async method and blocking method now(not only in hdfs). is that acceptable ?

@Xuanwo
Copy link
Member

Xuanwo commented Aug 23, 2024 •
edited
Loading

There's a lot of duplicate code between the async method and blocking method now(not only in hdfs). is that acceptable ?

Yes. This is rust (at 2024).

@meteorgan meteorgan force-pushed the fix_hdfs_write branch 3 times, most recently from 973f44e to 0ca5cf1 Compare August 25, 2024 01:17
Xuanwo
Xuanwo approved these changes Sep 3, 2024
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for fixing this!

@Xuanwo Xuanwo merged commit 782e66a into apache:main Sep 3, 2024
87 checks passed
@meteorgan meteorgan deleted the fix_hdfs_write branch September 20, 2024 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

@PsiACE PsiACE Awaiting requested review from PsiACE PsiACE is a code owner

Assignees
No one assigned
Labels
releases-note/fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

bug: fail to connect hdfs if set atomic_write_dir
2 participants
@meteorgan @Xuanwo