[Improvement] correct the behaviors when creating Iceberg table with none distribution #6196
Assignees
Labels
Comments
FANNG1
added a commit
that referenced
this issue
Jan 16, 2025
…thout specifying disribution mode (#6214) ### What changes were proposed in this pull request? Adjust the distribution mode for creating Iceberg table with none distribution. the following is the Spark adjust logic, the flink is similar. ```java private DistributionMode defaultWriteDistributionMode() { if (table.sortOrder().isSorted()) { return RANGE; } else if (table.spec().isPartitioned()) { return HASH; } else { return NONE; } } ``` ### Why are the changes needed? Fix: #6196 ### Does this PR introduce _any_ user-facing change? Yes, add document ### How was this patch tested? add UT and IT
github-actions bot
pushed a commit
that referenced
this issue
Jan 16, 2025
…thout specifying disribution mode (#6214) ### What changes were proposed in this pull request? Adjust the distribution mode for creating Iceberg table with none distribution. the following is the Spark adjust logic, the flink is similar. ```java private DistributionMode defaultWriteDistributionMode() { if (table.sortOrder().isSorted()) { return RANGE; } else if (table.spec().isPartitioned()) { return HASH; } else { return NONE; } } ``` ### Why are the changes needed? Fix: #6196 ### Does this PR introduce _any_ user-facing change? Yes, add document ### How was this patch tested? add UT and IT
jerryshao
pushed a commit
that referenced
this issue
Jan 16, 2025
…thout specifying disribution mode (#6299) ### What changes were proposed in this pull request? Adjust the distribution mode for creating Iceberg table with none distribution. the following is the Spark adjust logic, the flink is similar. ```java private DistributionMode defaultWriteDistributionMode() { if (table.sortOrder().isSorted()) { return RANGE; } else if (table.spec().isPartitioned()) { return HASH; } else { return NONE; } } ``` ### Why are the changes needed? Fix: #6196 ### Does this PR introduce _any_ user-facing change? Yes, add document ### How was this patch tested? add UT and IT Co-authored-by: FANNG <[email protected]>
jerqi
pushed a commit
that referenced
this issue
Jan 23, 2025
…thout specifying disribution mode (#6214) ### What changes were proposed in this pull request? Adjust the distribution mode for creating Iceberg table with none distribution. the following is the Spark adjust logic, the flink is similar. ```java private DistributionMode defaultWriteDistributionMode() { if (table.sortOrder().isSorted()) { return RANGE; } else if (table.spec().isPartitioned()) { return HASH; } else { return NONE; } } ``` ### Why are the changes needed? Fix: #6196 ### Does this PR introduce _any_ user-facing change? Yes, add document ### How was this patch tested? add UT and IT
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What would you like to be improved?
Gravitino
nonedistribution in API means the client side doesn't specify the distribution mode for the underlying catalog.Iceberg
nonedistribution means it doesn't distribute the data. Iceberg catalog transform Gravitino none distribution to Iceberg none distribution, it's not correct.How should we improve?
No response
The text was updated successfully, but these errors were encountered: