Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IcebergIO] cleanups, improvements, and test tweaks #33592

Merged
merged 2 commits into from
Jan 22, 2025

Conversation

ahmedabu98
Copy link
Contributor

Collection of some improvements:

  • cache Catalog instead of recreating it unnecessarily
  • use faster IO implementation org.apache.iceberg.gcp.gcs.GCSFileIO in tests
  • tests drop tables and namespaces right after completion
  • tests close resources properly

Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @damondouglas for label java.
R: @damccorm for label build.
R: @shunping for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Copy link
Contributor

Reminder, please take a look at this pr: @damondouglas @damccorm @shunping

Copy link
Contributor

@damondouglas damondouglas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@shunping shunping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ahmedabu98
Copy link
Contributor Author

Failing test is unrelated:

> Task :runners:google-cloud-dataflow-java:worker:test

org.apache.beam.runners.dataflow.worker.windmill.client.grpc.GrpcDirectGetWorkStreamTest > testConsumedWorkItems_itemsSplitAcrossResponses FAILED
    org.junit.runners.model.TestTimedOutException at GrpcDirectGetWorkStreamTest.java:130

@ahmedabu98 ahmedabu98 merged commit 0d3cfa9 into apache:master Jan 22, 2025
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants