Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement] refactor gravitino catalog to support FunctionCatalog #2903

Closed
FANNG1 opened this issue Apr 12, 2024 · 2 comments · Fixed by #2906
Closed

[Improvement] refactor gravitino catalog to support FunctionCatalog #2903

FANNG1 opened this issue Apr 12, 2024 · 2 comments · Fixed by #2906
Assignees
@FANNG1
Labels
improvement Improvements on everything
Milestone
Gravitino 0.5.0

Comments

@FANNG1
Copy link
Contributor

FANNG1 commented Apr 12, 2024

What would you like to be improved?

we must support FunctionCatalog in #2709, it's better to register specific catalog to Spark catalog manager, the specific catalog could implement FunctionCatalog

How should we improve?

#1227
@FANNG1 FANNG1 added the improvement Improvements on everything label Apr 12, 2024
@FANNG1 FANNG1 mentioned this issue Apr 12, 2024
Merged
@coolderli
Copy link
Contributor

@FANNG1 Does this mean that we will support Functions in Gravitino? So we can list functions and get function.

@FANNG1
Copy link
Contributor Author

FANNG1 commented Apr 12, 2024 •
edited
Loading

@FANNG1 Does this mean that we will support Functions in Gravitino? So we can list functions and get function.

No, supports functions in Gravitino are long term, this issue aims to refactor GravitinoCatalog to easily implement FunctionCatalog to support partitions for Iceberg. list and get functions will proxy to the underlying spark catalog not Gravitino server after this issue.

@jerryshao jerryshao added this to the Gravitino 0.5.0 milestone Apr 16, 2024
jerryshao pushed a commit that referenced this issue Apr 16, 2024
@FANNG1
[#2903]feat(spark-connector): register specific catalog to Spark cata…
49b07a2 
…log manager (#2906)

### What changes were proposed in this pull request?
register specific catalog like `GravitinoHiveCatalog` to Spark catalog
manager, no the general `GravitinoCatalog`

### Why are the changes needed?
The specific catalog could implement different interfaces like
`FunctionCatalog`, to support Iceberg partition.
Fix: #2903 

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
existing tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FANNG1 FANNG1

Labels
improvement Improvements on everything
Projects
None yet
Milestone
Gravitino 0.5.0
Development

Successfully merging a pull request may close this issue.

[#2903]feat(spark-connector): register specific catalog to Spark catalog manager FANNG1/gravitino
3 participants
@jerryshao @FANNG1 @coolderli