Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-34700][SQL] SessionCatalog's temporary view related APIs should take/return more concrete types #31906

Closed
wants to merge 8 commits into from

Conversation

imback82
Copy link
Contributor

What changes were proposed in this pull request?

Now that all the temporary views are wrapped with TemporaryViewRelation(#31273, #31652, and #31825), this PR proposes to update SessionCatalog's APIs for temporary views to take or return more concrete types.

APIs that will take TemporaryViewRelation instead of LogicalPlan:

createTempView, createGlobalTempView, alterTempViewDefinition

APIs that will return TemporaryViewRelation instead of LogicalPlan:

getRawTempView, getRawGlobalTempView

APIs that will return View instead of LogicalPlan:

getTempView, getGlobalTempView, lookupTempView

Why are the changes needed?

Internal refactoring to work with more concrete types.

Does this PR introduce any user-facing change?

No, this is internal refactoring.

How was this patch tested?

Updated existing tests affected by the refactoring.

}
}

def getTempViewSchema(plan: LogicalPlan): StructType = {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only called from SparkGetColumnsOperation.scala and now that a raw temp plan always has CatalogTable, the caller can just directly get the schema from the raw plan, and I don't see a reason to expose this as a def. Check SparkGetColumnsOperation.scala changes below.

@SparkQA
Copy link

SparkQA commented Mar 19, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40853/

@SparkQA
Copy link

SparkQA commented Mar 19, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40853/

@github-actions github-actions bot added the SQL label Mar 20, 2021
@SparkQA
Copy link

SparkQA commented Mar 20, 2021

Test build #136270 has finished for PR 31906 at commit fd9480c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@imback82
Copy link
Contributor Author

cc @cloud-fan

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in 7953fcd Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants