[SPARK-45328][SQL] Remove Hive support prior to 2.0.0 #43116
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dongjoon-hyun
approved these changes
Sep 26, 2023
pan3793
reviewed
Sep 26, 2023
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
Show resolved
Hide resolved
|
Should it be mentioned at |
pan3793
reviewed
Sep 26, 2023
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTempPath.scala
Outdated
Show resolved
Hide resolved
pan3793
reviewed
Sep 26, 2023
| @@ -1366,7 +1366,7 @@ private[client] class Shim_v1_1 extends Shim_v1_0 { | |||
|
|
|||
| } | |||
|
|
|||
| private[client] class Shim_v1_2 extends Shim_v1_1 { | |||
| private class Shim_v1_2 extends Shim_v1_1 { | |||
There was a problem hiding this comment.
It does not really help, would rather keep it as-is to reduce the git history
There was a problem hiding this comment.
Let's address them in your PR. Let me get this in first.
HyukjinKwon
force-pushed
the
SPARK-45328
branch
from
7738fab to
643efc7
Compare
|
Merged to master. |
HyukjinKwon
pushed a commit
that referenced
this pull request
Sep 30, 2023
### What changes were proposed in this pull request? This PR aims to clean up unused Hive shim classes for Hive prior 2.0.0, it's pure refactor, kind of something like `git squash` ### Why are the changes needed? SPARK-45328 removed Hive support prior to 2.0.0, this PR is the next step of the code clean up, see #43116 (comment) ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass GA ### Was this patch authored or co-authored using generative AI tooling? No. Closes #43151 from pan3793/SPARK-45358. Authored-by: Cheng Pan <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
MaxGekk
pushed a commit
that referenced
this pull request
Oct 27, 2023
…d versions ### What changes were proposed in this pull request? A minor follow-up of #43116 ### Why are the changes needed? Correct the docs of `spark.sql.hive.metastore.version` ### Does this PR introduce _any_ user-facing change? Yes, docs changed. ### How was this patch tested? Review ### Was this patch authored or co-authored using generative AI tooling? No Closes #43552 from pan3793/SPARK-45328-doc. Authored-by: Cheng Pan <[email protected]> Signed-off-by: Max Gekk <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR proposes to remove Hive support prior to 2.0.0 (
spark.sql.hive.metastore.version).Why are the changes needed?
We dropped JDK 8 and 11, and Hive prior to 2.0.0 cannot work together. They are actually already the dead code.
Does this PR introduce any user-facing change?
Technically no, because this wouldn't already work.
How was this patch tested?
Nope because there is no way to test them.
Was this patch authored or co-authored using generative AI tooling?
No.