-
Notifications
You must be signed in to change notification settings - Fork 28.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-34518][SQL] Rename AlterTableRecoverPartitionsCommand
to RepairTableCommand
#31635
Conversation
Test build #135413 has finished for PR 31635 at commit
|
@dongjoon-hyun @cloud-fan @HyukjinKwon Are you ok with the renaming? |
@@ -600,11 +600,11 @@ case class PartitionStatistics(numFiles: Int, totalSize: Long) | |||
* MSCK REPAIR TABLE table [{ADD|DROP|SYNC} PARTITIONS]; | |||
* }}} | |||
*/ | |||
case class AlterTableRecoverPartitionsCommand( | |||
case class RepairTableCommand( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit reluctant to rename v1 commands, as they are legacy and have been there for many years.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but this is non-public API. We have already extended AlterTableRecoverPartitionsCommand
in #31499. After that, it became incompatible by sources and binary with the previous version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And how about the logical nodes that were massively replaced in https://issues.apache.org/jira/browse/SPARK-29900. They are legacy and have been there for many years too, aren't they?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the point is that we might better focus on checking V2 instead of V1 commands because eventually V2 commands should replace V1 commands in the future. If this is likely only the one, I think it's fine. If there are a lot of such instances, I wouldn't change for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is likely only the one, I think it's fine.
Yep, only this for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm okay with this but let me leave it to @cloud-fan.
thanks, merging to master! |
+1, LGTM. |
What changes were proposed in this pull request?
Rename the execution node
AlterTableRecoverPartitionsCommand
for the commands:MSCK REPAIR TABLE table [{ADD|DROP|SYNC} PARTITIONS]
ALTER TABLE table RECOVER PARTITIONS
to
RepairTableCommand
.Why are the changes needed?
MSCK REPAIR TABLE .. [{ADD|DROP|SYNC} PARTITIONS]
#31499,ALTER TABLE table RECOVER PARTITIONS
is equal toMSCK REPAIR TABLE table ADD PARTITIONS
. And mapping of the generic commandMSCK REPAIR TABLE
to the more specific execution nodeAlterTableRecoverPartitionsCommand
can confuse devs in the future.ALTER TABLE table RECOVER PARTITIONS
does not support any options/extensions. So, additional parametersenableAddPartitions
andenableDropPartitions
inAlterTableRecoverPartitionsCommand
confuse as well.Does this PR introduce any user-facing change?
No because this is internal API.
How was this patch tested?
By running the existing test suites: