Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] Make Gravitino run with cloud storage #4396

Open
8 tasks
jerryshao opened this issue Aug 6, 2024 · 0 comments
Open
8 tasks

[EPIC] Make Gravitino run with cloud storage #4396

jerryshao opened this issue Aug 6, 2024 · 0 comments
Labels
epic Key feature

Comments

@jerryshao
Copy link
Contributor

Describe the proposal

This EPIC issue aims to make Gravitino support running with cloud storage, like S3, ADLS, GCS, etc.

The background is that current Gravitino only tests against HDFS, but with the more demands of cloud object storage, we should make sure that Gravitino can work well with cloud storage.

The work here includes necessary code changes, configuration supports, validations and tests.

Task list

  • Hive catalog supports cloud storage.
  • Iceberg catalog supports cloud storage
  • Fileset supports cloud storage
  • Iceberg rest catalog server supports cloud storage
  • Paimon supports cloud storage
  • Spark supports running on cloud storage
  • Flink supports running on cloud storage
  • Trino supports running on cloud storage
@jerryshao jerryshao added the epic Key feature label Aug 6, 2024
@jerryshao jerryshao added this to the Gravitino 0.7.0 milestone Aug 6, 2024
@jerryshao jerryshao removed this from the Gravitino 0.7.0 milestone Oct 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic Key feature
Projects
None yet
Development

No branches or pull requests

1 participant