From 5b7fba6bd90d990e41fdaeb1ea86e339c32443c3 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Wed, 27 Mar 2024 09:57:36 +0800 Subject: [PATCH 01/12] split mysql scripts --- build.gradle.kts | 1 + .../gravitino/storage/relational/TestRelationalEntityStore.java | 2 +- .../jdbc/h2/schema-0.5.0-h2.sql} | 0 docs/gravitino-server-config.md | 2 +- .../relational-entity-store/jdbc/mysql/schema-0.5.0-mysql.sql | 0 5 files changed, 3 insertions(+), 2 deletions(-) rename core/src/test/resources/{h2/h2-init.sql => relational-entity-store/jdbc/h2/schema-0.5.0-h2.sql} (100%) rename core/src/main/resources/mysql/mysql_init.sql => scripts/relational-entity-store/jdbc/mysql/schema-0.5.0-mysql.sql (100%) diff --git a/build.gradle.kts b/build.gradle.kts index b32b081b761..710f99a84d6 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -563,6 +563,7 @@ tasks { from(projectDir.dir("conf")) { into("package/conf") } from(projectDir.dir("bin")) { into("package/bin") } from(projectDir.dir("web/build/libs/${rootProject.name}-web-$version.war")) { into("package/web") } + from(projectDir.dir("scripts")) { into("package/scripts") } into(outputDir) rename { fileName -> fileName.replace(".template", "") diff --git a/core/src/test/java/com/datastrato/gravitino/storage/relational/TestRelationalEntityStore.java b/core/src/test/java/com/datastrato/gravitino/storage/relational/TestRelationalEntityStore.java index 58430308ff9..5afe3c9dd8e 100644 --- a/core/src/test/java/com/datastrato/gravitino/storage/relational/TestRelationalEntityStore.java +++ b/core/src/test/java/com/datastrato/gravitino/storage/relational/TestRelationalEntityStore.java @@ -93,7 +93,7 @@ public static void setUp() { entityStore.initialize(config); // Read the ddl sql to create table - String scriptPath = "h2/h2-init.sql"; + String scriptPath = "relational-entity-store/jdbc/h2/schema-0.5.0-h2.sql"; try (SqlSession sqlSession = SqlSessionFactoryHelper.getInstance().getSqlSessionFactory().openSession(true); Connection connection = sqlSession.getConnection(); diff --git a/core/src/test/resources/h2/h2-init.sql b/core/src/test/resources/relational-entity-store/jdbc/h2/schema-0.5.0-h2.sql similarity index 100% rename from core/src/test/resources/h2/h2-init.sql rename to core/src/test/resources/relational-entity-store/jdbc/h2/schema-0.5.0-h2.sql diff --git a/docs/gravitino-server-config.md b/docs/gravitino-server-config.md index 7150c7fa8d7..83bfb08166b 100644 --- a/docs/gravitino-server-config.md +++ b/docs/gravitino-server-config.md @@ -52,7 +52,7 @@ You can also specify filter parameters by setting configuration entries of the f | `gravitino.entity.store.maxTransactionSkewTimeMs` | The maximum skew time of transactions in milliseconds. | `2000` | No | 0.3.0 | | `gravitino.entity.store.kv.deleteAfterTimeMs` | The maximum time in milliseconds that deleted and old-version data is kept. Set to at least 10 minutes and no longer than 30 days. | `604800000`(7 days) | No | 0.3.0 | | `gravitino.entity.store.relational` | Detailed implementation of Relational storage. `MySQL` is currently supported, and the implementation is `JDBCBackend`. | `JDBCBackend` | No | 0.5.0 | -| `gravitino.entity.store.relational.jdbcUrl` | The database url that the `JDBCBackend` needs to connect to. If you use `MySQL`, you should firstly initialize the database tables yourself by executing the file named `mysql_init.sql` in the `src/main/resources/mysql` directory of the `core` module. | (none) | Yes if you use `JdbcBackend` | 0.5.0 | +| `gravitino.entity.store.relational.jdbcUrl` | The database url that the `JDBCBackend` needs to connect to. If you use `MySQL`, you should firstly initialize the database tables yourself by executing the ddl scripts in the `${GRAVITINO_HOME}/scripts/relational-entity-store/jdbc/mysql` directory. | (none) | Yes if you use `JdbcBackend` | 0.5.0 | | `gravitino.entity.store.relational.jdbcDriver` | The jdbc driver name that the `JDBCBackend` needs to use. You should place the driver Jar package in the `${GRAVITINO_HOME}/libs/` directory. | (none) | Yes if you use `JdbcBackend` | 0.5.0 | | `gravitino.entity.store.relational.jdbcUser` | The username that the `JDBCBackend` needs to use when connecting the database. It is required for `MySQL`. | (none) | Yes if you use `JdbcBackend` | 0.5.0 | | `gravitino.entity.store.relational.jdbcPassword` | The password that the `JDBCBackend` needs to use when connecting the database. It is required for `MySQL`. | (none) | Yes if you use `JdbcBackend` | 0.5.0 | diff --git a/core/src/main/resources/mysql/mysql_init.sql b/scripts/relational-entity-store/jdbc/mysql/schema-0.5.0-mysql.sql similarity index 100% rename from core/src/main/resources/mysql/mysql_init.sql rename to scripts/relational-entity-store/jdbc/mysql/schema-0.5.0-mysql.sql From 278c0e2c1ba1b828791c11646e2678bab015b4d8 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 1 Apr 2024 18:58:22 +0800 Subject: [PATCH 02/12] add docs --- docs/how-to-install.md | 3 +- docs/how-to-use-relational-entity-storage.md | 66 ++++++++++++++++++++ 2 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 docs/how-to-use-relational-entity-storage.md diff --git a/docs/how-to-install.md b/docs/how-to-install.md index 02d495c6af7..13321130a2f 100644 --- a/docs/how-to-install.md +++ b/docs/how-to-install.md @@ -43,7 +43,8 @@ The Gravitino binary distribution package contains the following files: | └── log4j2.properties # log4j configuration for the Gravitino server. |── libs/ # Gravitino server dependencies libraries. |── logs/ # Gravitino server logs. Automatically created after the Gravitino server starts. - └── data/ # Default directory for the Gravitino server to store data. + |── data/ # Default directory for the Gravitino server to store data. + └── scripts/ # Extra scripts for Gravitino. ``` #### Configure the Gravitino server diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md new file mode 100644 index 00000000000..7d0292dec44 --- /dev/null +++ b/docs/how-to-use-relational-entity-storage.md @@ -0,0 +1,66 @@ +--- +title: How to use relational entity storage +slug: /how-to-use-relational-entity-storage +license: "Copyright 2024 Datastrato Pvt Ltd. +This software is licensed under the Apache License version 2." +--- + +## Introduction + +Gravitino supports using `Relational Entity Storage` to store metadata after version `0.5.0`. Currently, you can use `MySQL` as the jdbc backend for `Relational Entity Storage`. + +## Steps for usage + +#### Prerequisites + ++ MySQL 5.7 or 8.0 ++ Gravitino distribution package ++ MySQL connector Jar (Should be compatible with your version of MySQL) + +#### Step 1: Get the initialization script + +You need `download` and `unzip` the distribution package firstly, please see: [How to install Gravitino](how-to-install.md). +Then you can get the initialization script in the directory: +```text +${distribution_package_directory}/scripts/relational-entity-store/jdbc/mysql/ +``` +The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. +For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. + +#### Step 2: Initialize the database + +Please `create a database` in MySQL in advance, and `execute` the initialization script obtained above in the database. + +#### Step 3: Place the MySQL connector Jar + +You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), which names like `mysql-connector-java-{driver-version}.jar`. +Then please place it in the distribution package directory: +```text +${distribution_package_directory}/libs/ +``` + +#### Step 4: Set up the Gravitino server configs + +Find the server configuration file names `gravitino.conf` in the distribution package directory: + +```text +${distribution_package_directory}/conf/ +``` +Then set up the following server configs: +```text +gravitino.entity.store = relational +gravitino.entity.store.relational = JDBCBackend +gravitino.entity.store.relational.jdbcUrl = ${your_jdbc_url} +gravitino.entity.store.relational.jdbcDriver = ${your_driver_name} +gravitino.entity.store.relational.jdbcUser = ${your_username} +gravitino.entity.store.relational.jdbcPassword = ${your_password} +``` + +#### Step 4: Start the server + +Finally, you can run the script in the distribution package directory to start the server: + +```shell +./${distribution_package_directory}/bin/gravitino.sh start +``` + From b73b68ab8beb3a8e4e29f99162f374a746cc7cd0 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 10:57:19 +0800 Subject: [PATCH 03/12] fix comments --- docs/how-to-use-relational-entity-storage.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index 7d0292dec44..09c21471a55 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -7,7 +7,16 @@ This software is licensed under the Apache License version 2." ## Introduction -Gravitino supports using `Relational Entity Storage` to store metadata after version `0.5.0`. Currently, you can use `MySQL` as the jdbc backend for `Relational Entity Storage`. +Gravitino supports using `Relational Entity Storage` to store metadata after version `0.5.0`. + +#### Target users +`Relational Entity Storage` is mainly aimed at users who are accustomed to using RDBMS to store data or lack available KV storage +to help them better to use Gravitino. + +#### Advantages +With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational +storage to manage metadata. +Currently, you can use `MySQL` as the jdbc backend for `Relational Entity Storage`. ## Steps for usage @@ -22,7 +31,7 @@ Gravitino supports using `Relational Entity Storage` to store metadata after ver You need `download` and `unzip` the distribution package firstly, please see: [How to install Gravitino](how-to-install.md). Then you can get the initialization script in the directory: ```text -${distribution_package_directory}/scripts/relational-entity-store/jdbc/mysql/ +${distribution_package_directory}/scripts/relational-entity-store/mysql/ ``` The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. From a199ef2a2b5cccbd2452657acec88fd4652a9c86 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 11:32:41 +0800 Subject: [PATCH 04/12] fix --- docs/how-to-use-relational-entity-storage.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index 09c21471a55..8d1a5e5776f 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -10,8 +10,7 @@ This software is licensed under the Apache License version 2." Gravitino supports using `Relational Entity Storage` to store metadata after version `0.5.0`. #### Target users -`Relational Entity Storage` is mainly aimed at users who are accustomed to using RDBMS to store data or lack available KV storage -to help them better to use Gravitino. +`Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. #### Advantages With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational From c5e0702d40eb8bc1e03bafcdb319c0b529bbb831 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 16:06:06 +0800 Subject: [PATCH 05/12] fix comments --- docs/how-to-use-relational-entity-storage.md | 34 ++++++++++++-------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index 8d1a5e5776f..a250711ded7 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -7,39 +7,45 @@ This software is licensed under the Apache License version 2." ## Introduction -Gravitino supports using `Relational Entity Storage` to store metadata after version `0.5.0`. +Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. -#### Target users +### What is Relational Entity Storage +Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata. +`Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. + +### Target users `Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. -#### Advantages -With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational -storage to manage metadata. -Currently, you can use `MySQL` as the jdbc backend for `Relational Entity Storage`. +### Advantages +With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational storage to manage metadata. + +### What kind of backend storage are supported +Currently, `Relational Entity Storage` supports the `JDBC Backend`, and it uses `MySQL` as the default storage for `JDBC Backend`. + ## Steps for usage -#### Prerequisites +### Prerequisites + MySQL 5.7 or 8.0 + Gravitino distribution package -+ MySQL connector Jar (Should be compatible with your version of MySQL) ++ MySQL connector Jar (Should be compatible with the version of MySQL instance) -#### Step 1: Get the initialization script +### Step 1: Get the initialization script You need `download` and `unzip` the distribution package firstly, please see: [How to install Gravitino](how-to-install.md). Then you can get the initialization script in the directory: ```text -${distribution_package_directory}/scripts/relational-entity-store/mysql/ +${distribution_package_directory}/scripts/mysql/ ``` The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. -#### Step 2: Initialize the database +### Step 2: Initialize the database Please `create a database` in MySQL in advance, and `execute` the initialization script obtained above in the database. -#### Step 3: Place the MySQL connector Jar +### Step 3: Place the MySQL connector Jar You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), which names like `mysql-connector-java-{driver-version}.jar`. Then please place it in the distribution package directory: @@ -47,7 +53,7 @@ Then please place it in the distribution package directory: ${distribution_package_directory}/libs/ ``` -#### Step 4: Set up the Gravitino server configs +### Step 4: Set up the Gravitino server configs Find the server configuration file names `gravitino.conf` in the distribution package directory: @@ -64,7 +70,7 @@ gravitino.entity.store.relational.jdbcUser = ${your_username} gravitino.entity.store.relational.jdbcPassword = ${your_password} ``` -#### Step 4: Start the server +### Step 5: Start the server Finally, you can run the script in the distribution package directory to start the server: From 84af519b70b45b84a6ecf497d06e891922ee10ef Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 16:15:05 +0800 Subject: [PATCH 06/12] fix --- docs/how-to-use-relational-entity-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index a250711ded7..2a8b0d21fa8 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -9,7 +9,7 @@ This software is licensed under the Apache License version 2." Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. -### What is Relational Entity Storage +### Overview Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata. `Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. From f61cbf5912dd96537260e5d898ba9bcc6b2dce6c Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 18:33:54 +0800 Subject: [PATCH 07/12] fix style --- docs/how-to-use-relational-entity-storage.md | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index 2a8b0d21fa8..9e2ba36dc7c 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -7,17 +7,17 @@ This software is licensed under the Apache License version 2." ## Introduction -Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. +Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. ### Overview Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata. `Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. ### Target users -`Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. +`Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. ### Advantages -With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational storage to manage metadata. +With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational storage to manage metadata. ### What kind of backend storage are supported Currently, `Relational Entity Storage` supports the `JDBC Backend`, and it uses `MySQL` as the default storage for `JDBC Backend`. @@ -34,11 +34,14 @@ Currently, `Relational Entity Storage` supports the `JDBC Backend`, and it uses ### Step 1: Get the initialization script You need `download` and `unzip` the distribution package firstly, please see: [How to install Gravitino](how-to-install.md). + Then you can get the initialization script in the directory: + ```text ${distribution_package_directory}/scripts/mysql/ ``` -The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. + +The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. ### Step 2: Initialize the database @@ -47,8 +50,11 @@ Please `create a database` in MySQL in advance, and `execute` the initialization ### Step 3: Place the MySQL connector Jar -You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), which names like `mysql-connector-java-{driver-version}.jar`. +You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), +which names like `mysql-connector-java-{driver-version}.jar`. + Then please place it in the distribution package directory: + ```text ${distribution_package_directory}/libs/ ``` @@ -60,7 +66,9 @@ Find the server configuration file names `gravitino.conf` in the distribution pa ```text ${distribution_package_directory}/conf/ ``` + Then set up the following server configs: + ```text gravitino.entity.store = relational gravitino.entity.store.relational = JDBCBackend @@ -78,3 +86,5 @@ Finally, you can run the script in the distribution package directory to start t ./${distribution_package_directory}/bin/gravitino.sh start ``` + + From db417a9185cdfdbc585163905e70827e9f9c5bd5 Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Mon, 8 Apr 2024 18:36:17 +0800 Subject: [PATCH 08/12] fix --- docs/how-to-use-relational-entity-storage.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md index 9e2ba36dc7c..dbfa28ac0e1 100644 --- a/docs/how-to-use-relational-entity-storage.md +++ b/docs/how-to-use-relational-entity-storage.md @@ -10,8 +10,7 @@ This software is licensed under the Apache License version 2." Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. ### Overview -Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata. -`Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. +Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata.`Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. ### Target users `Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. @@ -41,8 +40,7 @@ Then you can get the initialization script in the directory: ${distribution_package_directory}/scripts/mysql/ ``` -The script names like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. -For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. +The script name is like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. ### Step 2: Initialize the database @@ -51,7 +49,7 @@ Please `create a database` in MySQL in advance, and `execute` the initialization ### Step 3: Place the MySQL connector Jar You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), -which names like `mysql-connector-java-{driver-version}.jar`. +which is name like `mysql-connector-java-{driver-version}.jar`. Then please place it in the distribution package directory: @@ -61,7 +59,7 @@ ${distribution_package_directory}/libs/ ### Step 4: Set up the Gravitino server configs -Find the server configuration file names `gravitino.conf` in the distribution package directory: +Find the server configuration file which name is `gravitino.conf` in the distribution package directory: ```text ${distribution_package_directory}/conf/ From 37f95bf9c528965010dedd8de3a29af63fdca08e Mon Sep 17 00:00:00 2001 From: Jerry Shao Date: Mon, 8 Apr 2024 20:49:45 +0800 Subject: [PATCH 09/12] Polishing the doc --- docs/how-to-install.md | 5 + docs/how-to-use-relational-backend-storage.md | 92 +++++++++++++++++++ docs/how-to-use-relational-entity-storage.md | 88 ------------------ 3 files changed, 97 insertions(+), 88 deletions(-) create mode 100644 docs/how-to-use-relational-backend-storage.md delete mode 100644 docs/how-to-use-relational-entity-storage.md diff --git a/docs/how-to-install.md b/docs/how-to-install.md index 13321130a2f..de3ff653d5a 100644 --- a/docs/how-to-install.md +++ b/docs/how-to-install.md @@ -47,6 +47,11 @@ The Gravitino binary distribution package contains the following files: └── scripts/ # Extra scripts for Gravitino. ``` +#### Initialize the RDBMS (Optional) + +If you want to use the relational backend storage, you need to initialize the RDBMS firstly. For +the details on how to initialize the RDBMS, please check [How to use relational backend storage](./how-to-use-relational-backend-storage.md). + #### Configure the Gravitino server The Gravitino server configuration file is `conf/gravitino.conf`. You can configure the Gravitino diff --git a/docs/how-to-use-relational-backend-storage.md b/docs/how-to-use-relational-backend-storage.md new file mode 100644 index 00000000000..af2b0dffe2f --- /dev/null +++ b/docs/how-to-use-relational-backend-storage.md @@ -0,0 +1,92 @@ +--- +title: How to use relational backend storage +slug: /how-to-use-relational-backend-storage +license: "Copyright 2024 Datastrato Pvt Ltd. +This software is licensed under the Apache License version 2." +--- + +## Introduction + +Before the version `0.5.0`, Gravitino only supports KV backend storage to store metadata. Since +RDBMS is widely used in the industry, starting from the version `0.5.0`, Gravitino supports using +RDBMS as relational backend storage to store metadata. This doc will guide you on how to use the +relational backend storage in Gravitino. + +Relational backend storage mainly aims to the users who are accustomed to using `RDBMS` to +store data or lack available a KV storage, and want to use Gravitino. + +With relational backend storage, you can quickly deploy Gravitino in a production environment and +take advantage of relational storage to manage metadata. + +### What kind of backend storage are supported + +Currently, relational backend storage supports the `JDBCBackend`, and uses `MySQL` as the +default storage for `JDBC Backend`. + +## How to use + +### Prerequisites + ++ MySQL 5.7 or 8.0. ++ Gravitino distribution package. ++ MySQL connector Jar (Should be compatible with the version of MySQL instance). + +### Step 1: Get the initialization script + +You need `download` and `unzip` the distribution package firstly, please see [How to install Gravitino](how-to-install.md). + +Then you can get the initialization script in the directory: + +```text +${GRAVITINO_HOME}/scripts/mysql/ +``` + +The script name is like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. +For example, if your Gravitino version is `0.6.0`, then you can choose the **latest version** script +file that is equal or smaller than `0.6.0`. For example, you can choose the `schema-0.5.0-mysql.sql` script. + +### Step 2: Initialize the database + +Please `create a database` in MySQL in advance, and `execute` the initialization script obtained above in the database. + +### Step 3: Place the MySQL connector Jar + +You should **download** the MySQL connector Jar for the corresponding version of MySQL you use +(You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), +which is name like `mysql-connector-java-{version}.jar`. + +Then please place it in the distribution package directory: + +```text +${GRAVITINO_HOME}/libs/ +``` + +### Step 4: Set up the Gravitino server configs + +Find the server configuration file which name is `gravitino.conf` in the distribution package directory: + +```text +${GRAVITINO_HOME}/conf/ +``` + +Then set up the following server configs: + +```text +gravitino.entity.store = relational +gravitino.entity.store.relational = JDBCBackend +gravitino.entity.store.relational.jdbcUrl = ${your_jdbc_url} +gravitino.entity.store.relational.jdbcDriver = ${your_driver_name} +gravitino.entity.store.relational.jdbcUser = ${your_username} +gravitino.entity.store.relational.jdbcPassword = ${your_password} +``` + +### Step 5: Start the server + +Finally, you can run the script in the distribution package directory to start the server: + +```shell +./${GRAVITINO_HOME}/bin/gravitino.sh start +``` + + + diff --git a/docs/how-to-use-relational-entity-storage.md b/docs/how-to-use-relational-entity-storage.md deleted file mode 100644 index dbfa28ac0e1..00000000000 --- a/docs/how-to-use-relational-entity-storage.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -title: How to use relational entity storage -slug: /how-to-use-relational-entity-storage -license: "Copyright 2024 Datastrato Pvt Ltd. -This software is licensed under the Apache License version 2." ---- - -## Introduction - -Gravitino supports using `Relational Entity Storage` to store metadata after the version `0.5.0`. - -### Overview -Before the version `0.5.0`, Gravitino only supports `KV Entity Storage` to store metadata.`Relational Entity Storage` is an entity storage implementation that is compatible with `KV Entity Storage` at the interface layer and uses relational databases as the metadata backend storage. - -### Target users -`Relational Entity Storage` is mainly aimed at users who are accustomed to using `RDBMS` to store data or lack available a KV storage, and want to use Gravitino. - -### Advantages -With `Relational Entity Storage`, you can quickly deploy Gravitino in a production environment and take advantage of relational storage to manage metadata. - -### What kind of backend storage are supported -Currently, `Relational Entity Storage` supports the `JDBC Backend`, and it uses `MySQL` as the default storage for `JDBC Backend`. - - -## Steps for usage - -### Prerequisites - -+ MySQL 5.7 or 8.0 -+ Gravitino distribution package -+ MySQL connector Jar (Should be compatible with the version of MySQL instance) - -### Step 1: Get the initialization script - -You need `download` and `unzip` the distribution package firstly, please see: [How to install Gravitino](how-to-install.md). - -Then you can get the initialization script in the directory: - -```text -${distribution_package_directory}/scripts/mysql/ -``` - -The script name is like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.5.0`, then you should choose the `schema-0.5.0-mysql.sql` script. - -### Step 2: Initialize the database - -Please `create a database` in MySQL in advance, and `execute` the initialization script obtained above in the database. - -### Step 3: Place the MySQL connector Jar - -You should `download` the MySQL connector Jar for the corresponding version of MySQL you use (You can download it from the [maven-central-repo](https://repo1.maven.org/maven2/mysql/mysql-connector-java/)), -which is name like `mysql-connector-java-{driver-version}.jar`. - -Then please place it in the distribution package directory: - -```text -${distribution_package_directory}/libs/ -``` - -### Step 4: Set up the Gravitino server configs - -Find the server configuration file which name is `gravitino.conf` in the distribution package directory: - -```text -${distribution_package_directory}/conf/ -``` - -Then set up the following server configs: - -```text -gravitino.entity.store = relational -gravitino.entity.store.relational = JDBCBackend -gravitino.entity.store.relational.jdbcUrl = ${your_jdbc_url} -gravitino.entity.store.relational.jdbcDriver = ${your_driver_name} -gravitino.entity.store.relational.jdbcUser = ${your_username} -gravitino.entity.store.relational.jdbcPassword = ${your_password} -``` - -### Step 5: Start the server - -Finally, you can run the script in the distribution package directory to start the server: - -```shell -./${distribution_package_directory}/bin/gravitino.sh start -``` - - - From deb03d7f3f1a3e6bef84dceb6f89a279455091e7 Mon Sep 17 00:00:00 2001 From: Jerry Shao Date: Mon, 8 Apr 2024 20:51:32 +0800 Subject: [PATCH 10/12] remove empty lines --- docs/how-to-use-relational-backend-storage.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/how-to-use-relational-backend-storage.md b/docs/how-to-use-relational-backend-storage.md index af2b0dffe2f..8fcc2524c0e 100644 --- a/docs/how-to-use-relational-backend-storage.md +++ b/docs/how-to-use-relational-backend-storage.md @@ -87,6 +87,3 @@ Finally, you can run the script in the distribution package directory to start t ```shell ./${GRAVITINO_HOME}/bin/gravitino.sh start ``` - - - From 8f67948c0e943993f2a1c0a9c1e2640a791788ba Mon Sep 17 00:00:00 2001 From: xiaojiebao Date: Tue, 9 Apr 2024 10:34:13 +0800 Subject: [PATCH 11/12] fix --- docs/how-to-use-relational-backend-storage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-to-use-relational-backend-storage.md b/docs/how-to-use-relational-backend-storage.md index 8fcc2524c0e..9ddf2e92e2b 100644 --- a/docs/how-to-use-relational-backend-storage.md +++ b/docs/how-to-use-relational-backend-storage.md @@ -12,7 +12,7 @@ RDBMS is widely used in the industry, starting from the version `0.5.0`, Graviti RDBMS as relational backend storage to store metadata. This doc will guide you on how to use the relational backend storage in Gravitino. -Relational backend storage mainly aims to the users who are accustomed to using `RDBMS` to +Relational backend storage mainly aims to the users who are accustomed to using RDBMS to store data or lack available a KV storage, and want to use Gravitino. With relational backend storage, you can quickly deploy Gravitino in a production environment and @@ -21,7 +21,7 @@ take advantage of relational storage to manage metadata. ### What kind of backend storage are supported Currently, relational backend storage supports the `JDBCBackend`, and uses `MySQL` as the -default storage for `JDBC Backend`. +default storage for `JDBCBackend`. ## How to use From 1fbcbd0bc6140dd123914cd651fafee7e0e81b6f Mon Sep 17 00:00:00 2001 From: Jerry Shao Date: Tue, 9 Apr 2024 11:26:16 +0800 Subject: [PATCH 12/12] small changes --- docs/how-to-use-relational-backend-storage.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/how-to-use-relational-backend-storage.md b/docs/how-to-use-relational-backend-storage.md index 9ddf2e92e2b..3d6900cd13e 100644 --- a/docs/how-to-use-relational-backend-storage.md +++ b/docs/how-to-use-relational-backend-storage.md @@ -33,7 +33,8 @@ default storage for `JDBCBackend`. ### Step 1: Get the initialization script -You need `download` and `unzip` the distribution package firstly, please see [How to install Gravitino](how-to-install.md). +You need to `download` and `unzip` the distribution package firstly, please see +[How to install Gravitino](how-to-install.md). Then you can get the initialization script in the directory: @@ -43,11 +44,11 @@ ${GRAVITINO_HOME}/scripts/mysql/ The script name is like `schema-{version}-mysql.sql`, and the `version` depends on your Gravitino version. For example, if your Gravitino version is `0.6.0`, then you can choose the **latest version** script -file that is equal or smaller than `0.6.0`. For example, you can choose the `schema-0.5.0-mysql.sql` script. +file that is equal or smaller than `0.6.0`, you can choose the `schema-0.5.0-mysql.sql` script. ### Step 2: Initialize the database -Please `create a database` in MySQL in advance, and `execute` the initialization script obtained above in the database. +Please create a database in MySQL in advance, and execute the initialization script obtained above in the database. ### Step 3: Place the MySQL connector Jar