diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 77b620cc54f..ce11c9490dc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,9 +19,9 @@ # Contributing to Apache Gravitino -Thank you for your interest in contributing to Apache Gravitino! You are welcome to contribute in any way you can to enhance the project. Gravitino appreciates your assistance in making it better, whether through code contributions, documentation, tests, best practices, graphic design, or any other means that have a positive impact. +Thank you for your interest in contributing to Apache Gravitino! Your support is invaluable in enhancing the project, and we warmly welcome contributions of all kinds. Gravitino appreciates your assistance in making it better, whether through code contributions, documentation, tests, best practices, graphic design, or any other means that have a positive impact. -Before you get started, please read and follow these guidelines to ensure a smooth and productive collaboration. +To ensure a smooth and productive collaboration, please take a moment to review and follow our contribution guidelines before getting started. ## Table of contents @@ -82,7 +82,7 @@ To stop and start a local Gravitino server via `bin/gravitino.sh start` and `bin which java ``` -IntelliJ IDEA is an integrated development environment (IDE) for Java development. Setting the project SDK ensures that IntelliJ uses the correct Java version for building and running the project. +IntelliJ IDEA is an integrated development environment (IDE) for Java development. Setting the project SDK ensures that IntelliJ uses the correct Java version to build and run the project. You can open up WSL in the IntelliJ terminal. Find the down arrow and select ubuntu. @@ -94,9 +94,9 @@ You can open up WSL in the IntelliJ terminal. Find the down arrow and select ubu 1. Open VSCode extension marketplace, search for and install **WSL**. -2. On Windows, press `Ctrl+Shift+P` to open the command palette, and run `Shell Command: Install 'code' command in PATH`. +2. Press `Ctrl+Shift+P` to open the command palette, and run `Shell Command: Install 'code' command in PATH`. -Installing the WSL extension in VSCode allows you to open and edit files in your WSL environment directly from VSCode. Adding the `code` command to your PATH enables you to open VSCode from the WSL terminal. +Installing the WSL extension in VSCode allows you to open and edit files in your WSL environment directly from VSCode. Adding the `code` command to your PATH lets you open VSCode from the WSL terminal. #### Verify and Configure Environment Variables @@ -116,7 +116,7 @@ Installing the WSL extension in VSCode allows you to open and edit files in your C:\Users\epic\AppData\Local\Programs\Microsoft VS Code\bin ``` -Adding VSCode to the environment variables ensures that you can open VSCode from any command prompt or terminal window. +Adding VSCode to the environment variables ensures you can open VSCode from any command prompt or terminal window. **On Ubuntu (WSL):** @@ -148,7 +148,7 @@ Running `code --version` verifies that the `code` command is available. Using `c code . ``` - This command will open the current WSL directory in VSCode on Windows. If you haven't added `code` to your path, follow these steps: + This command will open the current WSL directory in VSCode. If you haven't added `code` to your path, follow these steps: - Open VSCode on Windows. @@ -158,7 +158,7 @@ Running `code --version` verifies that the `code` command is available. Using `c 3. **Ensure Remote - WSL is Active** - When VSCode opens, you should see a green bottom-left corner indicating that VSCode is connected to WSL. If it isn't, click on the green area and select `Remote-WSL: New Window` or `Remote-WSL: Reopen Folder in WSL`. + When VSCode opens, you should see a green bottom-left corner indicating that VSCode is connected to WSL. If not, click on the green area and select `Remote-WSL: New Window` or `Remote-WSL: Reopen Folder in WSL`. 4. **Edit and Develop Your Project** @@ -166,7 +166,7 @@ Running `code --version` verifies that the `code` command is available. Using `c ### Handling Memory Issues in WSL -If you ran into a memory issue when using WSL, here are some solutions to resolve it. +Here are some solutions if you encounter a memory issue when using WSL. 1. **Shut down WSL** If your WSL is open, you can shut it down in Windows PowerShell using the following command: @@ -177,7 +177,7 @@ If you ran into a memory issue when using WSL, here are some solutions to resolv 2. **Navigate to user folder** - Open up File Explorer and navigate to `C:\Users\`. + Open File Explorer and navigate to `C:\Users\`. 3. **Create the `.wslconfig` file** @@ -207,7 +207,7 @@ For project management policies, refer to [GOVERNANCE.md](GOVERNANCE.md). ### Future Development Directions -For future development directions, refer to the [ROADMAP.md](ROADMAP.md) document. +Refer to the [ROADMAP.md](ROADMAP.md) document for future development directions. ## Contribution guidelines @@ -217,7 +217,7 @@ Please read and follow the [Code of Conduct](CODE_OF_CONDUCT.md). Gravitino prov ### Reporting bugs -If you find a bug in Gravitino, please open an issue on GitHub. Be sure to include as much detail as possible, such as a clear description, steps to reproduce, and your environment. Please follow the template provided. If you encounter a security issue, please refer to [SECURITY.md](SECURITY.md). +If you find a bug in Gravitino, please open an issue on GitHub. Include as much detail as possible, such as a clear description, reproduction steps, and your environment. Please follow the template provided. If you encounter a security issue, please refer to [SECURITY.md](SECURITY.md). ### Suggesting enhancements @@ -259,32 +259,32 @@ For details on the review process, please refer to [MAINTAINERS.md](MAINTAINERS. ## Testing -The CI infrastructure runs unit and integration tests on each pull request, please make sure these tests pass before making a pull request. +The CI infrastructure runs unit and integration tests on each pull request. Please make sure these tests pass before making a pull request. -The unit tests run on every build and integration tests run as needed. See [how to test](docs/how-to-test.md) for more information. +The unit tests run on every build, and integration tests run as needed. See [how to test](docs/how-to-test.md) for more information. -When adding new code or fixing a bug be sure to add unit tests to provide coverage. +Add unit tests to provide coverage when adding new code or fixing a bug. ## Coding standards -Spotless checks code formatting. If your code isn't correctly formatted, the build fails. To correctly format your code please use Spotless. +Spotless checks code formatting. If your code is correctly formatted, the build succeeds. To format your code correctly, please use Spotless. ```bash ./gradlew spotlessApply ``` -All files must have a license header and the build fails if any files are missing license headers. If you are adding third-party code be sure to understand how to add the third-party license to Gravitino LICENSE and NOTICE files. +All files must have a license header, and the build fails if any files are missing license headers. If you are adding third-party code, you may need to add the third-party license to Gravitino's LICENSE and NOTICE files. If you are unsure of how to do this, please ask for help. -For any bugs or new code please add unit tests to provide coverage of the code. The project may not accept code without unit tests. +Please add unit tests to provide code coverage for any bugs or new code. The project may not accept code without unit tests. -All text files should use macOS/unix style line endings (LF) not windows style line endings (CRLF). +All text files should use macOS/unix style line endings (LF), not Windows-style line endings (CRLF). ## Community and communication -Join the [community mailing list](https://lists.apache.org/list.html?dev@gravitino.apache.org) to discuss ideas and seek help and are also encouraged to use GitHub discussions. +Join the [community mailing list](https://lists.apache.org/list.html?dev@gravitino.apache.org) to discuss ideas and seek help. You are also encouraged to use GitHub discussions. ## License -When contributing to this project, you agree that your contributions use the Apache License version 2. Please ensure you have permission to do this if required by your employer. +When contributing to this project, you agree that your contributions use the Apache License version 2. Please ensure you have permission to do this if your employer requires it. Thank you for your contributions to Gravitino! The project appreciates your help in making it a success. diff --git a/build.gradle.kts b/build.gradle.kts index d290bd61eb3..100b243b54d 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -771,6 +771,7 @@ tasks { subprojects.forEach() { if (!it.name.startsWith("catalog") && !it.name.startsWith("authorization") && + !it.name.startsWith("cli") && !it.name.startsWith("client") && !it.name.startsWith("filesystem") && !it.name.startsWith("spark") && !it.name.startsWith("iceberg") && it.name != "trino-connector" && it.name != "integration-test" && it.name != "bundled-catalog" && !it.name.startsWith("flink") && it.name != "integration-test" && it.name != "hive-metastore-common" && !it.name.startsWith("flink") && diff --git a/clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/GravitinoVirtualFileSystem.java b/clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/GravitinoVirtualFileSystem.java index aaa81ab556b..a1e7bb5d55a 100644 --- a/clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/GravitinoVirtualFileSystem.java +++ b/clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/GravitinoVirtualFileSystem.java @@ -25,12 +25,14 @@ import com.github.benmanes.caffeine.cache.Scheduler; import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Preconditions; +import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.util.concurrent.ThreadFactoryBuilder; import java.io.File; import java.io.IOException; import java.net.URI; import java.util.Arrays; +import java.util.List; import java.util.Map; import java.util.concurrent.ScheduledThreadPoolExecutor; import java.util.concurrent.ThreadFactory; @@ -57,6 +59,8 @@ import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.fs.permission.FsPermission; +import org.apache.hadoop.security.Credentials; +import org.apache.hadoop.security.token.Token; import org.apache.hadoop.util.Progressable; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -559,6 +563,19 @@ public long getDefaultBlockSize(Path f) { return context.getFileSystem().getDefaultBlockSize(context.getActualFileLocation()); } + @Override + public Token[] addDelegationTokens(String renewer, Credentials credentials) { + List> tokenList = Lists.newArrayList(); + for (FileSystem fileSystem : internalFileSystemCache.asMap().values()) { + try { + tokenList.addAll(Arrays.asList(fileSystem.addDelegationTokens(renewer, credentials))); + } catch (IOException e) { + Logger.warn("Failed to add delegation tokens for filesystem: {}", fileSystem.getUri(), e); + } + } + return tokenList.stream().distinct().toArray(Token[]::new); + } + @Override public synchronized void close() throws IOException { // close all actual FileSystems diff --git a/core/src/main/java/org/apache/gravitino/listener/CatalogEventDispatcher.java b/core/src/main/java/org/apache/gravitino/listener/CatalogEventDispatcher.java index ff70d4cffda..782ef7df4e9 100644 --- a/core/src/main/java/org/apache/gravitino/listener/CatalogEventDispatcher.java +++ b/core/src/main/java/org/apache/gravitino/listener/CatalogEventDispatcher.java @@ -33,14 +33,19 @@ import org.apache.gravitino.exceptions.NonEmptyEntityException; import org.apache.gravitino.listener.api.event.AlterCatalogEvent; import org.apache.gravitino.listener.api.event.AlterCatalogFailureEvent; +import org.apache.gravitino.listener.api.event.AlterCatalogPreEvent; import org.apache.gravitino.listener.api.event.CreateCatalogEvent; import org.apache.gravitino.listener.api.event.CreateCatalogFailureEvent; +import org.apache.gravitino.listener.api.event.CreateCatalogPreEvent; import org.apache.gravitino.listener.api.event.DropCatalogEvent; import org.apache.gravitino.listener.api.event.DropCatalogFailureEvent; +import org.apache.gravitino.listener.api.event.DropCatalogPreEvent; import org.apache.gravitino.listener.api.event.ListCatalogEvent; import org.apache.gravitino.listener.api.event.ListCatalogFailureEvent; +import org.apache.gravitino.listener.api.event.ListCatalogPreEvent; import org.apache.gravitino.listener.api.event.LoadCatalogEvent; import org.apache.gravitino.listener.api.event.LoadCatalogFailureEvent; +import org.apache.gravitino.listener.api.event.LoadCatalogPreEvent; import org.apache.gravitino.listener.api.info.CatalogInfo; import org.apache.gravitino.utils.PrincipalUtils; @@ -68,6 +73,7 @@ public CatalogEventDispatcher(EventBus eventBus, CatalogDispatcher dispatcher) { @Override public NameIdentifier[] listCatalogs(Namespace namespace) throws NoSuchMetalakeException { + eventBus.dispatchEvent(new ListCatalogPreEvent(PrincipalUtils.getCurrentUserName(), namespace)); try { NameIdentifier[] nameIdentifiers = dispatcher.listCatalogs(namespace); eventBus.dispatchEvent(new ListCatalogEvent(PrincipalUtils.getCurrentUserName(), namespace)); @@ -81,6 +87,7 @@ public NameIdentifier[] listCatalogs(Namespace namespace) throws NoSuchMetalakeE @Override public Catalog[] listCatalogsInfo(Namespace namespace) throws NoSuchMetalakeException { + eventBus.dispatchEvent(new ListCatalogPreEvent(PrincipalUtils.getCurrentUserName(), namespace)); try { Catalog[] catalogs = dispatcher.listCatalogsInfo(namespace); eventBus.dispatchEvent(new ListCatalogEvent(PrincipalUtils.getCurrentUserName(), namespace)); @@ -94,6 +101,7 @@ public Catalog[] listCatalogsInfo(Namespace namespace) throws NoSuchMetalakeExce @Override public Catalog loadCatalog(NameIdentifier ident) throws NoSuchCatalogException { + eventBus.dispatchEvent(new LoadCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident)); try { Catalog catalog = dispatcher.loadCatalog(ident); eventBus.dispatchEvent( @@ -115,6 +123,10 @@ public Catalog createCatalog( String comment, Map properties) throws NoSuchMetalakeException, CatalogAlreadyExistsException { + CatalogInfo catalogInfo = + new CatalogInfo(ident.name(), type, provider, comment, properties, null); + eventBus.dispatchEvent( + new CreateCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident, catalogInfo)); try { Catalog catalog = dispatcher.createCatalog(ident, type, provider, comment, properties); eventBus.dispatchEvent( @@ -134,6 +146,8 @@ public Catalog createCatalog( @Override public Catalog alterCatalog(NameIdentifier ident, CatalogChange... changes) throws NoSuchCatalogException, IllegalArgumentException { + eventBus.dispatchEvent( + new AlterCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident, changes)); try { Catalog catalog = dispatcher.alterCatalog(ident, changes); eventBus.dispatchEvent( @@ -150,6 +164,7 @@ public Catalog alterCatalog(NameIdentifier ident, CatalogChange... changes) @Override public boolean dropCatalog(NameIdentifier ident, boolean force) throws NonEmptyEntityException, CatalogInUseException { + eventBus.dispatchEvent(new DropCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident)); try { boolean isExists = dispatcher.dropCatalog(ident, force); eventBus.dispatchEvent( diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/AlterCatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/AlterCatalogPreEvent.java new file mode 100644 index 00000000000..a8786adf48f --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/AlterCatalogPreEvent.java @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.CatalogChange; +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents an event triggered before altering a catalog. */ +@DeveloperApi +public final class AlterCatalogPreEvent extends CatalogPreEvent { + private final CatalogChange[] catalogChanges; + + public AlterCatalogPreEvent( + String user, NameIdentifier identifier, CatalogChange[] catalogChanges) { + super(user, identifier); + this.catalogChanges = catalogChanges; + } + + /** + * Retrieves the specific changes that were made to the catalog during the alteration process. + * + * @return An array of {@link CatalogChange} objects detailing each modification applied to the + * catalog. + */ + public CatalogChange[] catalogChanges() { + return catalogChanges; + } +} diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/CatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/CatalogPreEvent.java new file mode 100644 index 00000000000..e1fcd51cb17 --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/CatalogPreEvent.java @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents a pre-event for catalog operations. */ +@DeveloperApi +public abstract class CatalogPreEvent extends PreEvent { + protected CatalogPreEvent(String user, NameIdentifier identifier) { + super(user, identifier); + } +} diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/CreateCatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/CreateCatalogPreEvent.java new file mode 100644 index 00000000000..9abc1cbbfee --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/CreateCatalogPreEvent.java @@ -0,0 +1,46 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.listener.api.info.CatalogInfo; + +/** Represents an event triggered before creating a catalog. */ +@DeveloperApi +public class CreateCatalogPreEvent extends CatalogPreEvent { + private final CatalogInfo createCatalogRequest; + + public CreateCatalogPreEvent( + String user, NameIdentifier identifier, CatalogInfo createCatalogRequest) { + super(user, identifier); + this.createCatalogRequest = createCatalogRequest; + } + + /** + * Retrieves the create catalog request. + * + * @return A {@link CatalogInfo} instance encapsulating the comprehensive details of create + * catalog request. + */ + public CatalogInfo createCatalogRequest() { + return createCatalogRequest; + } +} diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/DropCatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/DropCatalogPreEvent.java new file mode 100644 index 00000000000..f359c99e712 --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/DropCatalogPreEvent.java @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents an event that is generated before a catalog is dropped. */ +@DeveloperApi +public final class DropCatalogPreEvent extends CatalogPreEvent { + public DropCatalogPreEvent(String user, NameIdentifier identifier) { + super(user, identifier); + } +} diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/ListCatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/ListCatalogPreEvent.java new file mode 100644 index 00000000000..d430b80ac08 --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/ListCatalogPreEvent.java @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.Namespace; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents an event that is triggered before listing of catalogs within a namespace. */ +@DeveloperApi +public final class ListCatalogPreEvent extends CatalogPreEvent { + private final Namespace namespace; + + public ListCatalogPreEvent(String user, Namespace namespace) { + super(user, NameIdentifier.of(namespace.levels())); + this.namespace = namespace; + } + + /** + * Provides the namespace associated with this event. + * + * @return A {@link Namespace} instance from which catalogs were listed. + */ + public Namespace namespace() { + return namespace; + } +} diff --git a/core/src/main/java/org/apache/gravitino/listener/api/event/LoadCatalogPreEvent.java b/core/src/main/java/org/apache/gravitino/listener/api/event/LoadCatalogPreEvent.java new file mode 100644 index 00000000000..b73552be933 --- /dev/null +++ b/core/src/main/java/org/apache/gravitino/listener/api/event/LoadCatalogPreEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents an event triggered before loading a catalog. */ +@DeveloperApi +public final class LoadCatalogPreEvent extends CatalogPreEvent { + + public LoadCatalogPreEvent(String user, NameIdentifier identifier) { + super(user, identifier); + } +} diff --git a/core/src/test/java/org/apache/gravitino/listener/api/event/TestCatalogEvent.java b/core/src/test/java/org/apache/gravitino/listener/api/event/TestCatalogEvent.java index 7b7cb9e3c46..b1ac4eba659 100644 --- a/core/src/test/java/org/apache/gravitino/listener/api/event/TestCatalogEvent.java +++ b/core/src/test/java/org/apache/gravitino/listener/api/event/TestCatalogEvent.java @@ -71,6 +71,12 @@ void testCreateCatalogEvent() { Assertions.assertEquals(CreateCatalogEvent.class, event.getClass()); CatalogInfo catalogInfo = ((CreateCatalogEvent) event).createdCatalogInfo(); checkCatalogInfo(catalogInfo, catalog); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(identifier, preEvent.identifier()); + Assertions.assertEquals(CreateCatalogPreEvent.class, preEvent.getClass()); + CatalogInfo preCatalogInfo = ((CreateCatalogPreEvent) preEvent).createCatalogRequest(); + checkCatalogInfo(preCatalogInfo, catalog); } @Test @@ -82,6 +88,10 @@ void testLoadCatalogEvent() { Assertions.assertEquals(LoadCatalogEvent.class, event.getClass()); CatalogInfo catalogInfo = ((LoadCatalogEvent) event).loadedCatalogInfo(); checkCatalogInfo(catalogInfo, catalog); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(identifier, preEvent.identifier()); + Assertions.assertEquals(LoadCatalogPreEvent.class, preEvent.getClass()); } @Test @@ -97,6 +107,13 @@ void testAlterCatalogEvent() { CatalogChange[] catalogChanges = ((AlterCatalogEvent) event).catalogChanges(); Assertions.assertEquals(1, catalogChanges.length); Assertions.assertEquals(catalogChange, catalogChanges[0]); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(identifier, preEvent.identifier()); + Assertions.assertEquals(AlterCatalogPreEvent.class, preEvent.getClass()); + CatalogChange[] preCatalogChanges = ((AlterCatalogPreEvent) preEvent).catalogChanges(); + Assertions.assertEquals(1, preCatalogChanges.length); + Assertions.assertEquals(catalogChange, preCatalogChanges[0]); } @Test @@ -107,6 +124,10 @@ void testDropCatalogEvent() { Assertions.assertEquals(identifier, event.identifier()); Assertions.assertEquals(DropCatalogEvent.class, event.getClass()); Assertions.assertEquals(true, ((DropCatalogEvent) event).isExists()); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(identifier, preEvent.identifier()); + Assertions.assertEquals(DropCatalogPreEvent.class, preEvent.getClass()); } @Test @@ -117,6 +138,11 @@ void testListCatalogEvent() { Assertions.assertEquals(namespace.toString(), event.identifier().toString()); Assertions.assertEquals(ListCatalogEvent.class, event.getClass()); Assertions.assertEquals(namespace, ((ListCatalogEvent) event).namespace()); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(namespace.toString(), preEvent.identifier().toString()); + Assertions.assertEquals(ListCatalogPreEvent.class, preEvent.getClass()); + Assertions.assertEquals(namespace, ((ListCatalogPreEvent) preEvent).namespace()); } @Test @@ -127,6 +153,11 @@ void testListCatalogInfoEvent() { Assertions.assertEquals(namespace.toString(), event.identifier().toString()); Assertions.assertEquals(ListCatalogEvent.class, event.getClass()); Assertions.assertEquals(namespace, ((ListCatalogEvent) event).namespace()); + + PreEvent preEvent = dummyEventListener.popPreEvent(); + Assertions.assertEquals(namespace.toString(), preEvent.identifier().toString()); + Assertions.assertEquals(ListCatalogPreEvent.class, preEvent.getClass()); + Assertions.assertEquals(namespace, ((ListCatalogPreEvent) preEvent).namespace()); } @Test diff --git a/docs/gravitino-server-config.md b/docs/gravitino-server-config.md index ca27bf39b32..68f5dda69bd 100644 --- a/docs/gravitino-server-config.md +++ b/docs/gravitino-server-config.md @@ -129,11 +129,12 @@ Gravitino triggers a pre-event before the operation, a post-event after the comp ##### Pre-event -| Operation type | Pre-event | Since Version | -|-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------| -| Iceberg REST server table operation | `IcebergCreateTablePreEvent`, `IcebergUpdateTablePreEvent`, `IcebergDropTablePreEvent`, `IcebergLoadTablePreEvent`, `IcebergListTablePreEvent`, `IcebergTableExistsPreEvent`, `IcebergRenameTablePreEvent` | 0.7.0-incubating | -| Gravitino server table operation | `CreateTablePreEvent`, `AlterTablePreEvent`, `DropTablePreEvent`, `PurgeTablePreEvent`, `LoadTablePreEvent`, `ListTablePreEvent` | 0.8.0-incubating | -| Gravitino server schema operation | `CreareSchemaPreEvent`, `AlterSchemaPreEvent`, `DropSchemaPreEvent`, `LoadSchemaPreEvent`, `ListSchemaPreEvent` | 0.8.0-incubating | +| Operation type | Pre-event | Since Version | +|-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| Iceberg REST server table operation | `IcebergCreateTablePreEvent`, `IcebergUpdateTablePreEvent`, `IcebergDropTablePreEvent`, `IcebergLoadTablePreEvent`, `IcebergListTablePreEvent`, `IcebergTableExistsPreEvent`, `IcebergRenameTablePreEvent` | 0.7.0-incubating | +| Gravitino server table operation | `CreateTablePreEvent`, `AlterTablePreEvent`, `DropTablePreEvent`, `PurgeTablePreEvent`, `LoadTablePreEvent`, `ListTablePreEvent` | 0.8.0-incubating | +| Gravitino server schema operation | `CreateSchemaPreEvent`, `AlterSchemaPreEvent`, `DropSchemaPreEvent`, `LoadSchemaPreEvent`, `ListSchemaPreEvent` | 0.8.0-incubating | +| Gravitino server catalog operation | `CreateCatalogPreEvent`, `AlterCatalogPreEvent`, `DropCatalogPreEvent`, `LoadCatalogPreEvent`, `ListCatalogPreEvent` | 0.8.0-incubating | | Gravitino server partition operation| `AddPartitionPreEvent`, `DropPartitionPreEvent`, `GetPartitionPreEvent`, `PurgePartitionPreEvent`,`ListPartitionPreEvent`,`ListPartitionNamesPreEvent` | 0.8.0-incubating | #### Event listener plugin diff --git a/docs/how-to-build.md b/docs/how-to-build.md index f929a840c90..a15b324169b 100644 --- a/docs/how-to-build.md +++ b/docs/how-to-build.md @@ -18,159 +18,134 @@ license: "This software is licensed under the Apache License version 2." :::info Please read the following notes before trying to build Gravitino. -+ Gravitino requires a minimum of JDK8, and supports up to JDK17 to run Gradle, so you need to install a JDK, version 8 to 17, to launch the build environment. -+ Gravitino itself supports using JDK8, 11, or 17 to build. The Gravitino Trino connector uses - JDK17 to build (to avoid vendor-related issues on some platforms, Gravitino uses the specified Amazon Corretto OpenJDK 17 to build the Trino connector on macOS). - You don't have to preinstall the specified JDK environment, as Gradle detects the JDK version needed and downloads it automatically. -+ Gravitino uses the Gradle Java Toolchain to detect and manage JDK versions, and it checks the - installed JDK by running the `./gradlew javaToolchains` command. See [Gradle Java Toolchain](https://docs.gradle.org/current/userguide/toolchains.html#sec:java_toolchain). -+ Gravitino excludes all Docker-related tests by default. To run Docker-related tests, make sure you have installed - Docker in your environment and either (1) set `skipDockerTests=false` in the `gradle.properties` file (or - use `-PskipDockerTests=false` in the command) or (2) `export SKIP_DOCKER_TESTS=false` in shell. Otherwise, all tests requiring Docker will be skipped. -+ macOS uses `docker-connector` to make the Gravitino Trino connector work with Docker - for macOS. See [docker-connector](https://github.com/wenjunxiao/mac-docker-connector), `$GRAVITINO_HOME/dev/docker/tools/mac-docker-connector.sh`, and - `$GRAVITINO_HOME/dev/docker/tools/README.md` for more details. -+ You can use OrbStack as a replacement for Docker for macOS. See - [OrbStack](https://orbstack.dev/). With OrbStack, you can run Gravitino integration tests - without needing to install `docker-connector`. ++ Gravitino requires a minimum of JDK8 and supports up to JDK17 to run Gradle, so you need to install a JDK, version 8 to 17, to launch the build environment. ++ Gravitino itself supports using JDK 8, 11, or 17 to build. The Gravitino Trino connector uses JDK17 to build (to avoid vendor-related issues on some platforms, Gravitino uses the specified Amazon Corretto OpenJDK 17 to build the Trino connector on macOS). + You don't have to preinstall the specified JDK environment, as Gradle detects the JDK version needed and downloads it automatically. ++ Gravitino uses the Gradle Java Toolchain to detect and manage JDK versions, and it checks the installed JDK by running the `./gradlew javaToolchains` command. See [Gradle Java Toolchain](https://docs.gradle.org/current/userguide/toolchains.html#sec:java_toolchain). ++ Gravitino excludes all Docker-related tests by default. To run Docker-related tests, make sure you have installed Docker in your environment and either (1) set `skipDockerTests=false` in the `gradle.properties` file (or use `-PskipDockerTests=false` in the command) or (2) `export SKIP_DOCKER_TESTS=false` in the shell. Otherwise, all tests requiring Docker will be skipped. ++ macOS uses `docker-connector` to make the Gravitino Trino connector work with Docker for macOS. See [docker-connector](https://github.com/wenjunxiao/mac-docker-connector), `$GRAVITINO_HOME/dev/docker/tools/mac-docker-connector.sh`, and `$GRAVITINO_HOME/dev/docker/tools/README.md` for more details. ++ You can use OrbStack as a replacement for Docker for macOS. See [OrbStack](https://orbstack.dev/). With OrbStack, you can run Gravitino integration tests without needing to install `docker-connector`. ::: ## Quick start 1. Clone the Gravitino project. - If you want to contribute to this open-source project, please fork the project on GitHub first. After forking, clone the forked project to your local environment, make your changes, and submit a pull request (PR). + If you want to contribute to this open-source project, please fork the project on GitHub first. After forking, clone the forked project to your local environment, make your changes, and submit a pull request (PR). - ```shell - git clone git@github.com:apache/gravitino.git - ``` + ```shell + git clone git@github.com:apache/gravitino.git + ``` 2. Build the Gravitino project. Running this for the first time can take 15 minutes or more. - ```shell - cd gravitino - ./gradlew build - ``` + ```shell + cd gravitino + ./gradlew build + ``` - The default specified JDK version is 8, but if you want to use JDK 11 or 17 to build, - modify the property `jdkVersion` to 11 or 17 in the `gradle.properties` file, or specify the version - with `-P`, like: + The default specified JDK version is 8, but if you want to use JDK 11 or 17 to build, modify the property `jdkVersion` to 11 or 17 in the `gradle.properties` file, or specify the version with `-P`, like: - ```shell - ./gradlew build -PjdkVersion=11 - ``` + ```shell + ./gradlew build -PjdkVersion=11 + ``` - Or: + Or: - ```shell - ./gradlew build -PjdkVersion=17 - ``` + ```shell + ./gradlew build -PjdkVersion=17 + ``` - The `./gradlew build` command builds all the Gravitino components, including Gravitino - server, Java and Python clients, Trino and Spark connectors, and more. + The `./gradlew build` command builds all the Gravitino components, including the Gravitino server, Java and Python clients, Trino and Spark connectors, and more. - For the Python client, the `./gradlew build` command builds the Python client with Python 3.8 - by default. If you want to use Python 3.9, 3.10 or 3.11 to build, please modify the property - `pythonVersion` to 3.9, 3.10 or 3.11 in the `gradle.properties` file, or specify the version - with `-P`, like: + For the Python client, the `./gradlew build` command builds the Python client with Python 3.8 by default. If you want to use Python 3.9, 3.10, or 3.11 to build, please modify the property `pythonVersion` to 3.9, 3.10, or 3.11 in the `gradle.properties` file, or specify the version with `-P` like: - ```shell - ./gradlew build -PpythonVersion=3.9 - ``` + ```shell + ./gradlew build -PpythonVersion=3.9 + ``` - Or: + Or: - ```shell - ./gradlew build -PpythonVersion=3.10 - ``` + ```shell + ./gradlew build -PpythonVersion=3.10 + ``` - Or: + Or: + + ```shell + ./gradlew build -PpythonVersion=3.11 + ``` - ```shell - ./gradlew build -PpythonVersion=3.11 - ``` - - If you want to build some modules alone, like Spark connector, you can use Gradle build task - with specific module name, like: - - ```shell - ./gradlew spark-connector:spark-runtime-3.4:build -PscalaVersion=2.12 - ``` - - This creates `gravitino-spark-connector-runtime-{sparkVersion}_{scalaVersion}-{version}.jar` - under the `spark-connector/v3.4/spark-runtime/build/libs` directory. You could replace `3.4` with - `3.3` or `3.5` to specify different Spark versions, and replace `2.12` with `2.13` for different Scala - versions. The default Scala version is `2.12` if not specifying `-PscalaVersion`. - - :::info - Gravitino Spark connector doesn't support Scala 2.13 for Spark 3.3. - ::: - - :::note - The first time you build the project, downloading the dependencies may take a while. You can add - `-x test` to skip the tests using `./gradlew build -x test`. - - The built Gravitino libraries are Java 8 compatible and verified under Java 8, 11, and 17 - environments. You can use Java 8, 11, or 17 runtimes to run the Gravitino server, no matter which - JDK version you used to build the project. - - The built jars are under the `build/libs` directory of the modules. You can publish them to your - Maven repository to use them in your project. - ::: + If you want to build a module on its own, like the Spark connector, you can use Gradle to build a module with a specific name, like so: -3. Get the Gravitino server binary package. + ```shell + ./gradlew spark-connector:spark-runtime-3.4:build -PscalaVersion=2.12 + ``` - ```shell - ./gradlew compileDistribution - ``` + This creates `gravitino-spark-connector-runtime-{sparkVersion}_{scalaVersion}-{version}.jar` under the `spark-connector/v3.4/spark-runtime/build/libs` directory. You could replace `3.4` with `3.3` or `3.5` to specify different Spark versions and replace `2.12` with `2.13` for different Scala versions. The default Scala version is `2.12` if `-PscalaVersion` is not specified. - The `compileDistribution` command creates a `distribution` directory in the Gravitino root directory. + :::info + Gravitino Spark connector doesn't support Scala 2.13 for Spark 3.3. + ::: - The directory structure of the `distribution` directory is as follows: + :::note + The first time you build the project, downloading the dependencies may take a while. + + You can add `-x test` to skip the tests using `./gradlew build -x test`. - :::note - The `./gradlew clean` command deletes the `distribution` directory. - ::: + The built Gravitino libraries are Java 8 compatible and verified under the Java 8, 11, and 17 environments. You can use Java 8, 11, or 17 runtimes to run the Gravitino server, no matter which JDK version was used to build the project. + + The built jars are under the modules `build/libs` directory. You can publish them in your Maven repository for use in your project. + ::: + +3. Get the Gravitino server binary package. + + ```shell + ./gradlew compileDistribution + ``` + + The `compileDistribution` command creates a `distribution` directory in the Gravitino root directory. + + :::note + The `./gradlew clean` command deletes the `distribution` directory. + ::: 4. Assemble the Gravitino server distribution package. - ```shell - ./gradlew assembleDistribution - ``` + ```shell + ./gradlew assembleDistribution + ``` - The `assembleDistribution` command creates `gravitino-{version}-bin.tar.gz` and `gravitino-{version}-bin.tar.gz.sha256` under the `distribution` directory. + The `assembleDistribution` command creates `gravitino-{version}-bin.tar.gz` and `gravitino-{version}-bin.tar.gz.sha256` under the `distribution` directory. - You can deploy them to your production environment. + You can deploy these to your production environment. - :::note - The `gravitino-{version}-bin.tar.gz` file is the Gravitino **server** distribution package, and the - `gravitino-{version}-bin.tar.gz.sha256` file is the sha256 checksum file for the Gravitino - server distribution package. - ::: + :::note + The `gravitino-{version}-bin.tar.gz` file is the Gravitino **server** distribution package, and the `gravitino-{version}-bin.tar.gz.sha256` file is the sha256 checksum file for the Gravitino server distribution package. + ::: 5. Assemble the Gravitino Trino connector package - ```shell - ./gradlew assembleTrinoConnector - ``` + ```shell + ./gradlew assembleTrinoConnector + ``` - or + or - ```shell - ./gradlew assembleDistribution - ``` + ```shell + ./gradlew assembleDistribution + ``` - This creates `gravitino-trino-connector-{version}.tar.gz` and - `gravitino-trino-connector-{version}.tar.gz.sha256` under the `distribution` directory. You - can uncompress and deploy it to Trino to use the Gravitino Trino connector. + This creates `gravitino-trino-connector-{version}.tar.gz` and + `gravitino-trino-connector-{version}.tar.gz.sha256` under the `distribution` directory. You can uncompress and deploy it to Trino to use the Gravitino Trino connector. 6. Assemble the Gravitino Iceberg REST server package - ```shell - ./gradlew assembleIcebergRESTServer - ``` + ```shell + ./gradlew assembleIcebergRESTServer + ``` - This creates `gravitino-iceberg-rest-server-{version}.tar.gz` and `gravitino-iceberg-rest-server-{version}.tar.gz.sha256` under the `distribution` directory. You can uncompress and deploy it to use the Gravitino Iceberg REST server. + This creates `gravitino-iceberg-rest-server-{version}.tar.gz` and `gravitino-iceberg-rest-server-{version}.tar.gz.sha256` under the `distribution` directory. You can uncompress and deploy it to use the Gravitino Iceberg REST server. ## How to build Apache Gravitino on Windows (Using WSL) @@ -180,7 +155,7 @@ license: "This software is licensed under the Apache License version 2." Refer to this guide for installation: [WSL Installation Guide](https://learn.microsoft.com/en-us/windows/wsl/install) -*Note: Ubuntu 22.04 can successfully run Gravitino* +*Note: Gravitino can run successfully on Ubuntu 22.04* This step involves setting up your Windows machine's Windows Subsystem for Linux (WSL). WSL allows you to run a Linux distribution alongside Windows, providing a Linux-like environment for development. @@ -201,22 +176,22 @@ Updating the package list ensures you have the latest information on the newest 1. Edit your `~/.bashrc` file using any editor. Here, `vim` is used: - ```shell - vim ~/.bashrc - ``` + ```shell + vim ~/.bashrc + ``` 2. Add the following lines at the end of the file. Replace `/usr/lib/jvm/java-11-openjdk-amd64` with your actual Java installation path: - ```sh - export JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64" - export PATH=$PATH:$JAVA_HOME/bin - ``` + ```sh + export JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64" + export PATH=$PATH:$JAVA_HOME/bin + ``` 3. Save and quit in vim using `:wq`. 4. Run `source ~/.bashrc` to update your shell session's environment variables. - Editing the `~/.bashrc` file allows you to set environment variables available in every terminal session. Setting `JAVA_HOME` and updating `PATH` ensures that your system uses the correct Java version for development. + Editing the `~/.bashrc` file allows you to set environment variables available in every terminal session. Setting `JAVA_HOME` and updating `PATH` ensures that your system uses the correct Java version for development. ### Install Docker diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/RESTService.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/RESTService.java index b301204bd20..d8fdc26f32b 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/RESTService.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/RESTService.java @@ -31,6 +31,9 @@ import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableEventDispatcher; import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableOperationDispatcher; import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableOperationExecutor; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewEventDispatcher; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationExecutor; import org.apache.gravitino.iceberg.service.metrics.IcebergMetricsManager; import org.apache.gravitino.iceberg.service.provider.IcebergConfigProvider; import org.apache.gravitino.iceberg.service.provider.IcebergConfigProviderFactory; @@ -90,6 +93,10 @@ private void initServer(IcebergConfig icebergConfig) { new IcebergTableOperationExecutor(icebergCatalogWrapperManager); IcebergTableEventDispatcher icebergTableEventDispatcher = new IcebergTableEventDispatcher(icebergTableOperationExecutor, eventBus, metalakeName); + IcebergViewOperationExecutor icebergViewOperationExecutor = + new IcebergViewOperationExecutor(icebergCatalogWrapperManager); + IcebergViewEventDispatcher icebergViewEventDispatcher = + new IcebergViewEventDispatcher(icebergViewOperationExecutor, eventBus, metalakeName); config.register( new AbstractBinder() { @@ -98,6 +105,7 @@ protected void configure() { bind(icebergCatalogWrapperManager).to(IcebergCatalogWrapperManager.class).ranked(1); bind(icebergMetricsManager).to(IcebergMetricsManager.class).ranked(1); bind(icebergTableEventDispatcher).to(IcebergTableOperationDispatcher.class).ranked(1); + bind(icebergViewEventDispatcher).to(IcebergViewOperationDispatcher.class).ranked(1); } }); diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewEventDispatcher.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewEventDispatcher.java new file mode 100644 index 00000000000..89137ea9129 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewEventDispatcher.java @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.iceberg.service.dispatcher; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.gravitino.listener.EventBus; +import org.apache.gravitino.listener.api.event.IcebergCreateViewEvent; +import org.apache.gravitino.listener.api.event.IcebergCreateViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergCreateViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergRequestContext; +import org.apache.gravitino.listener.api.event.IcebergViewExistsEvent; +import org.apache.gravitino.listener.api.event.IcebergViewExistsFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergViewExistsPreEvent; +import org.apache.iceberg.catalog.Namespace; +import org.apache.iceberg.catalog.TableIdentifier; +import org.apache.iceberg.rest.requests.CreateViewRequest; +import org.apache.iceberg.rest.requests.RenameTableRequest; +import org.apache.iceberg.rest.requests.UpdateTableRequest; +import org.apache.iceberg.rest.responses.ListTablesResponse; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +/** + * {@code IcebergViewEventDispatcher} is a decorator for {@link IcebergViewOperationExecutor} that + * not only delegates view operations to the underlying dispatcher but also dispatches corresponding + * events to an {@link EventBus}. + */ +public class IcebergViewEventDispatcher implements IcebergViewOperationDispatcher { + + private IcebergViewOperationDispatcher icebergViewOperationDispatcher; + private EventBus eventBus; + private String metalakeName; + + public IcebergViewEventDispatcher( + IcebergViewOperationDispatcher icebergViewOperationDispatcher, + EventBus eventBus, + String metalakeName) { + this.icebergViewOperationDispatcher = icebergViewOperationDispatcher; + this.eventBus = eventBus; + this.metalakeName = metalakeName; + } + + @Override + public LoadViewResponse createView( + IcebergRequestContext context, Namespace namespace, CreateViewRequest createViewRequest) { + TableIdentifier viewIdentifier = TableIdentifier.of(namespace, createViewRequest.name()); + NameIdentifier nameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), viewIdentifier); + eventBus.dispatchEvent( + new IcebergCreateViewPreEvent(context, nameIdentifier, createViewRequest)); + LoadViewResponse loadViewResponse; + try { + loadViewResponse = + icebergViewOperationDispatcher.createView(context, namespace, createViewRequest); + } catch (Exception e) { + eventBus.dispatchEvent( + new IcebergCreateViewFailureEvent(context, nameIdentifier, createViewRequest, e)); + throw e; + } + eventBus.dispatchEvent( + new IcebergCreateViewEvent(context, nameIdentifier, createViewRequest, loadViewResponse)); + return loadViewResponse; + } + + @Override + public LoadViewResponse replaceView( + IcebergRequestContext context, + TableIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest) { + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), viewIdentifier); + eventBus.dispatchEvent( + new IcebergReplaceViewPreEvent(context, gravitinoNameIdentifier, replaceViewRequest)); + LoadViewResponse loadViewResponse; + try { + loadViewResponse = + icebergViewOperationDispatcher.replaceView(context, viewIdentifier, replaceViewRequest); + } catch (Exception e) { + eventBus.dispatchEvent( + new IcebergReplaceViewFailureEvent( + context, gravitinoNameIdentifier, replaceViewRequest, e)); + throw e; + } + eventBus.dispatchEvent( + new IcebergReplaceViewEvent( + context, gravitinoNameIdentifier, replaceViewRequest, loadViewResponse)); + return loadViewResponse; + } + + @Override + public void dropView(IcebergRequestContext context, TableIdentifier viewIdentifier) { + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), viewIdentifier); + eventBus.dispatchEvent(new IcebergDropViewPreEvent(context, gravitinoNameIdentifier)); + try { + icebergViewOperationDispatcher.dropView(context, viewIdentifier); + } catch (Exception e) { + eventBus.dispatchEvent(new IcebergDropViewFailureEvent(context, gravitinoNameIdentifier, e)); + throw e; + } + eventBus.dispatchEvent(new IcebergDropViewEvent(context, gravitinoNameIdentifier)); + } + + @Override + public LoadViewResponse loadView(IcebergRequestContext context, TableIdentifier viewIdentifier) { + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), viewIdentifier); + eventBus.dispatchEvent(new IcebergLoadViewPreEvent(context, gravitinoNameIdentifier)); + LoadViewResponse loadViewResponse; + try { + loadViewResponse = icebergViewOperationDispatcher.loadView(context, viewIdentifier); + } catch (Exception e) { + eventBus.dispatchEvent(new IcebergLoadViewFailureEvent(context, gravitinoNameIdentifier, e)); + throw e; + } + eventBus.dispatchEvent( + new IcebergLoadViewEvent(context, gravitinoNameIdentifier, loadViewResponse)); + return loadViewResponse; + } + + @Override + public ListTablesResponse listView(IcebergRequestContext context, Namespace namespace) { + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier(metalakeName, context.catalogName(), namespace); + eventBus.dispatchEvent(new IcebergListViewPreEvent(context, gravitinoNameIdentifier)); + ListTablesResponse listViewsResponse; + try { + listViewsResponse = icebergViewOperationDispatcher.listView(context, namespace); + } catch (Exception e) { + eventBus.dispatchEvent(new IcebergListViewFailureEvent(context, gravitinoNameIdentifier, e)); + throw e; + } + eventBus.dispatchEvent(new IcebergListViewEvent(context, gravitinoNameIdentifier)); + return listViewsResponse; + } + + @Override + public boolean viewExists(IcebergRequestContext context, TableIdentifier viewIdentifier) { + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), viewIdentifier); + eventBus.dispatchEvent(new IcebergViewExistsPreEvent(context, gravitinoNameIdentifier)); + boolean isExists; + try { + isExists = icebergViewOperationDispatcher.viewExists(context, viewIdentifier); + } catch (Exception e) { + eventBus.dispatchEvent( + new IcebergViewExistsFailureEvent(context, gravitinoNameIdentifier, e)); + throw e; + } + eventBus.dispatchEvent(new IcebergViewExistsEvent(context, gravitinoNameIdentifier, isExists)); + return isExists; + } + + @Override + public void renameView(IcebergRequestContext context, RenameTableRequest renameViewRequest) { + TableIdentifier sourceView = renameViewRequest.source(); + NameIdentifier gravitinoNameIdentifier = + IcebergRestUtils.getGravitinoNameIdentifier( + metalakeName, context.catalogName(), sourceView); + eventBus.dispatchEvent( + new IcebergRenameViewPreEvent(context, gravitinoNameIdentifier, renameViewRequest)); + try { + icebergViewOperationDispatcher.renameView(context, renameViewRequest); + } catch (Exception e) { + eventBus.dispatchEvent( + new IcebergRenameViewFailureEvent( + context, gravitinoNameIdentifier, renameViewRequest, e)); + throw e; + } + eventBus.dispatchEvent( + new IcebergRenameViewEvent(context, gravitinoNameIdentifier, renameViewRequest)); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationDispatcher.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationDispatcher.java new file mode 100644 index 00000000000..2f084a744ce --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationDispatcher.java @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.iceberg.service.dispatcher; + +import org.apache.gravitino.listener.api.event.IcebergRequestContext; +import org.apache.iceberg.catalog.Namespace; +import org.apache.iceberg.catalog.TableIdentifier; +import org.apache.iceberg.rest.requests.CreateViewRequest; +import org.apache.iceberg.rest.requests.RenameTableRequest; +import org.apache.iceberg.rest.requests.UpdateTableRequest; +import org.apache.iceberg.rest.responses.ListTablesResponse; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +/** + * The {@code IcebergViewOperationDispatcher} interface defines the public API for managing Iceberg + * views. + */ +public interface IcebergViewOperationDispatcher { + + /** + * Creates a new Iceberg view. + * + * @param context Iceberg REST request context information. + * @param namespace The namespace within which the view should be created. + * @param createViewRequest The request object containing the details for creating the view. + * @return A {@link LoadViewResponse} object containing the result of the operation. + */ + LoadViewResponse createView( + IcebergRequestContext context, Namespace namespace, CreateViewRequest createViewRequest); + + /** + * Updates an Iceberg view. + * + * @param context Iceberg REST request context information. + * @param viewIdentifier The Iceberg view identifier. + * @param replaceViewRequest The request object containing the details for updating the view. + * @return A {@link LoadViewResponse} object containing the result of the operation. + */ + LoadViewResponse replaceView( + IcebergRequestContext context, + TableIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest); + + /** + * Drops an Iceberg view. + * + * @param context Iceberg REST request context information. + * @param viewIdentifier The Iceberg view identifier. + */ + void dropView(IcebergRequestContext context, TableIdentifier viewIdentifier); + + /** + * Loads an Iceberg view. + * + * @param context Iceberg REST request context information. + * @param viewIdentifier The Iceberg view identifier. + * @return A {@link LoadViewResponse} object containing the result of the operation. + */ + LoadViewResponse loadView(IcebergRequestContext context, TableIdentifier viewIdentifier); + + /** + * Lists Iceberg views. + * + * @param context Iceberg REST request context information. + * @param namespace The Iceberg namespace. + * @return A {@link ListTablesResponse} object containing the list of view identifiers. + */ + ListTablesResponse listView(IcebergRequestContext context, Namespace namespace); + + /** + * Check whether an Iceberg view exists. + * + * @param context Iceberg REST request context information. + * @param viewIdentifier The Iceberg view identifier. + * @return Whether view exists. + */ + boolean viewExists(IcebergRequestContext context, TableIdentifier viewIdentifier); + + /** + * Rename an Iceberg view. + * + * @param context Iceberg REST request context information. + * @param renameViewRequest Rename view request information. + */ + void renameView(IcebergRequestContext context, RenameTableRequest renameViewRequest); +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationExecutor.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationExecutor.java new file mode 100644 index 00000000000..308c20c437c --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/dispatcher/IcebergViewOperationExecutor.java @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.iceberg.service.dispatcher; + +import org.apache.gravitino.iceberg.service.IcebergCatalogWrapperManager; +import org.apache.gravitino.listener.api.event.IcebergRequestContext; +import org.apache.iceberg.catalog.Namespace; +import org.apache.iceberg.catalog.TableIdentifier; +import org.apache.iceberg.rest.requests.CreateViewRequest; +import org.apache.iceberg.rest.requests.RenameTableRequest; +import org.apache.iceberg.rest.requests.UpdateTableRequest; +import org.apache.iceberg.rest.responses.ListTablesResponse; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +public class IcebergViewOperationExecutor implements IcebergViewOperationDispatcher { + + private IcebergCatalogWrapperManager icebergCatalogWrapperManager; + + public IcebergViewOperationExecutor(IcebergCatalogWrapperManager icebergCatalogWrapperManager) { + this.icebergCatalogWrapperManager = icebergCatalogWrapperManager; + } + + @Override + public LoadViewResponse createView( + IcebergRequestContext context, Namespace namespace, CreateViewRequest createViewRequest) { + return icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .createView(namespace, createViewRequest); + } + + @Override + public LoadViewResponse replaceView( + IcebergRequestContext context, + TableIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest) { + return icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .updateView(viewIdentifier, replaceViewRequest); + } + + @Override + public void dropView(IcebergRequestContext context, TableIdentifier viewIdentifier) { + icebergCatalogWrapperManager.getCatalogWrapper(context.catalogName()).dropView(viewIdentifier); + } + + @Override + public LoadViewResponse loadView(IcebergRequestContext context, TableIdentifier viewIdentifier) { + return icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .loadView(viewIdentifier); + } + + @Override + public ListTablesResponse listView(IcebergRequestContext context, Namespace namespace) { + return icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .listView(namespace); + } + + @Override + public boolean viewExists(IcebergRequestContext context, TableIdentifier viewIdentifier) { + return icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .existView(viewIdentifier); + } + + @Override + public void renameView(IcebergRequestContext context, RenameTableRequest renameViewRequest) { + icebergCatalogWrapperManager + .getCatalogWrapper(context.catalogName()) + .renameView(renameViewRequest); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableOperations.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableOperations.java index 3ad5fa7ff4b..67dcfce023d 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableOperations.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableOperations.java @@ -249,7 +249,7 @@ public Response tableExists( TableIdentifier tableIdentifier = TableIdentifier.of(icebergNS, table); boolean exists = tableOperationDispatcher.tableExists(context, tableIdentifier); if (exists) { - return IcebergRestUtils.okWithoutContent(); + return IcebergRestUtils.noContent(); } else { return IcebergRestUtils.notExists(); } diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableRenameOperations.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableRenameOperations.java index bd2021d74d8..23acc1da124 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableRenameOperations.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergTableRenameOperations.java @@ -68,7 +68,7 @@ public Response renameTable( renameTableRequest.destination()); IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); tableOperationDispatcher.renameTable(context, renameTableRequest); - return IcebergRestUtils.okWithoutContent(); + return IcebergRestUtils.noContent(); } // HTTP request is null in Jersey test, override with a mock request when testing. diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewOperations.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewOperations.java index 3e46257e22b..26c466504a9 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewOperations.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewOperations.java @@ -20,6 +20,9 @@ import com.codahale.metrics.annotation.ResponseMetered; import com.codahale.metrics.annotation.Timed; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.google.common.annotations.VisibleForTesting; import javax.inject.Inject; import javax.servlet.http.HttpServletRequest; import javax.ws.rs.Consumes; @@ -33,30 +36,37 @@ import javax.ws.rs.core.Context; import javax.ws.rs.core.MediaType; import javax.ws.rs.core.Response; -import org.apache.gravitino.iceberg.service.IcebergCatalogWrapperManager; +import org.apache.gravitino.iceberg.service.IcebergObjectMapper; import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; +import org.apache.gravitino.listener.api.event.IcebergRequestContext; import org.apache.gravitino.metrics.MetricNames; +import org.apache.iceberg.catalog.Namespace; import org.apache.iceberg.catalog.TableIdentifier; import org.apache.iceberg.rest.RESTUtil; import org.apache.iceberg.rest.requests.CreateViewRequest; import org.apache.iceberg.rest.requests.UpdateTableRequest; import org.apache.iceberg.rest.responses.ListTablesResponse; import org.apache.iceberg.rest.responses.LoadViewResponse; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; @Path("/v1/{prefix:([^/]*/)?}namespaces/{namespace}/views") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.APPLICATION_JSON) public class IcebergViewOperations { - private IcebergCatalogWrapperManager icebergCatalogWrapperManager; + private static final Logger LOG = LoggerFactory.getLogger(IcebergViewOperations.class); - @SuppressWarnings("UnusedVariable") - @Context - private HttpServletRequest httpRequest; + private ObjectMapper icebergObjectMapper; + private IcebergViewOperationDispatcher viewOperationDispatcher; + + @Context private HttpServletRequest httpRequest; @Inject - public IcebergViewOperations(IcebergCatalogWrapperManager icebergCatalogWrapperManager) { - this.icebergCatalogWrapperManager = icebergCatalogWrapperManager; + public IcebergViewOperations(IcebergViewOperationDispatcher viewOperationDispatcher) { + this.viewOperationDispatcher = viewOperationDispatcher; + this.icebergObjectMapper = IcebergObjectMapper.getInstance(); } @GET @@ -65,9 +75,12 @@ public IcebergViewOperations(IcebergCatalogWrapperManager icebergCatalogWrapperM @ResponseMetered(name = "list-view", absolute = true) public Response listView( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace) { - ListTablesResponse response = - icebergCatalogWrapperManager.getOps(prefix).listView(RESTUtil.decodeNamespace(namespace)); - return IcebergRestUtils.ok(response); + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info("List Iceberg views, catalog: {}, namespace: {}", catalogName, icebergNS); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + ListTablesResponse listTablesResponse = viewOperationDispatcher.listView(context, icebergNS); + return IcebergRestUtils.ok(listTablesResponse); } @POST @@ -77,12 +90,19 @@ public Response listView( public Response createView( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace, - CreateViewRequest request) { - LoadViewResponse response = - icebergCatalogWrapperManager - .getOps(prefix) - .createView(RESTUtil.decodeNamespace(namespace), request); - return IcebergRestUtils.ok(response); + CreateViewRequest createViewRequest) { + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info( + "Create Iceberg view, catalog: {}, namespace: {}, createViewRequest: {}", + catalogName, + icebergNS, + createViewRequest); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + LoadViewResponse loadViewResponse = + viewOperationDispatcher.createView(context, icebergNS, createViewRequest); + + return IcebergRestUtils.ok(loadViewResponse); } @GET @@ -94,10 +114,15 @@ public Response loadView( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace, @PathParam("view") String view) { - TableIdentifier viewIdentifier = TableIdentifier.of(RESTUtil.decodeNamespace(namespace), view); - LoadViewResponse response = - icebergCatalogWrapperManager.getOps(prefix).loadView(viewIdentifier); - return IcebergRestUtils.ok(response); + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info( + "Load Iceberg view, catalog: {}, namespace: {}, view: {}", catalogName, icebergNS, view); + + TableIdentifier viewIdentifier = TableIdentifier.of(icebergNS, view); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + LoadViewResponse loadViewResponse = viewOperationDispatcher.loadView(context, viewIdentifier); + return IcebergRestUtils.ok(loadViewResponse); } @POST @@ -109,11 +134,20 @@ public Response replaceView( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace, @PathParam("view") String view, - UpdateTableRequest request) { - TableIdentifier viewIdentifier = TableIdentifier.of(RESTUtil.decodeNamespace(namespace), view); - LoadViewResponse response = - icebergCatalogWrapperManager.getOps(prefix).updateView(viewIdentifier, request); - return IcebergRestUtils.ok(response); + UpdateTableRequest replaceViewRequest) { + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info( + "Replace Iceberg view, catalog: {}, namespace: {}, table: {}, replaceViewRequest: {}", + catalogName, + icebergNS, + view, + SerializeReplaceViewRequest(replaceViewRequest)); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + TableIdentifier viewIdentifier = TableIdentifier.of(icebergNS, view); + LoadViewResponse loadViewResponse = + viewOperationDispatcher.replaceView(context, viewIdentifier, replaceViewRequest); + return IcebergRestUtils.ok(loadViewResponse); } @DELETE @@ -125,8 +159,13 @@ public Response dropView( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace, @PathParam("view") String view) { - TableIdentifier viewIdentifier = TableIdentifier.of(RESTUtil.decodeNamespace(namespace), view); - icebergCatalogWrapperManager.getOps(prefix).dropView(viewIdentifier); + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info( + "Drop Iceberg view, catalog: {}, namespace: {}, table: {}", catalogName, icebergNS, view); + TableIdentifier viewIdentifier = TableIdentifier.of(namespace, view); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + viewOperationDispatcher.dropView(context, viewIdentifier); return IcebergRestUtils.noContent(); } @@ -139,11 +178,35 @@ public Response viewExists( @PathParam("prefix") String prefix, @PathParam("namespace") String namespace, @PathParam("view") String view) { - TableIdentifier tableIdentifier = TableIdentifier.of(RESTUtil.decodeNamespace(namespace), view); - if (icebergCatalogWrapperManager.getOps(prefix).existView(tableIdentifier)) { + String catalogName = IcebergRestUtils.getCatalogName(prefix); + Namespace icebergNS = RESTUtil.decodeNamespace(namespace); + LOG.info( + "Check Iceberg view exists, catalog: {}, namespace: {}, table: {}", + catalogName, + icebergNS, + view); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + TableIdentifier viewIdentifier = TableIdentifier.of(icebergNS, view); + boolean exists = viewOperationDispatcher.viewExists(context, viewIdentifier); + if (exists) { return IcebergRestUtils.noContent(); } else { return IcebergRestUtils.notExists(); } } + + // HTTP request is null in Jersey test, override with a mock request when testing. + @VisibleForTesting + HttpServletRequest httpServletRequest() { + return httpRequest; + } + + private String SerializeReplaceViewRequest(UpdateTableRequest replaceViewRequest) { + try { + return icebergObjectMapper.writeValueAsString(replaceViewRequest); + } catch (JsonProcessingException e) { + LOG.warn("Serialize update view request failed", e); + return replaceViewRequest.toString(); + } + } } diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewRenameOperations.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewRenameOperations.java index 128689d33be..4c6f706086b 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewRenameOperations.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/iceberg/service/rest/IcebergViewRenameOperations.java @@ -20,6 +20,7 @@ import com.codahale.metrics.annotation.ResponseMetered; import com.codahale.metrics.annotation.Timed; +import com.google.common.annotations.VisibleForTesting; import javax.inject.Inject; import javax.servlet.http.HttpServletRequest; import javax.ws.rs.Consumes; @@ -30,33 +31,49 @@ import javax.ws.rs.core.Context; import javax.ws.rs.core.MediaType; import javax.ws.rs.core.Response; -import org.apache.gravitino.iceberg.service.IcebergCatalogWrapperManager; import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; +import org.apache.gravitino.listener.api.event.IcebergRequestContext; import org.apache.gravitino.metrics.MetricNames; import org.apache.iceberg.rest.requests.RenameTableRequest; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; @Path("/v1/{prefix:([^/]*/)?}views/rename") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.APPLICATION_JSON) public class IcebergViewRenameOperations { + private static final Logger LOG = LoggerFactory.getLogger(IcebergViewRenameOperations.class); - @SuppressWarnings("UnusedVariable") - @Context - private HttpServletRequest httpRequest; + @Context private HttpServletRequest httpRequest; - private IcebergCatalogWrapperManager icebergCatalogWrapperManager; + private IcebergViewOperationDispatcher viewOperationDispatcher; @Inject - public IcebergViewRenameOperations(IcebergCatalogWrapperManager icebergCatalogWrapperManager) { - this.icebergCatalogWrapperManager = icebergCatalogWrapperManager; + public IcebergViewRenameOperations(IcebergViewOperationDispatcher viewOperationDispatcher) { + this.viewOperationDispatcher = viewOperationDispatcher; } @POST @Produces(MediaType.APPLICATION_JSON) @Timed(name = "rename-view." + MetricNames.HTTP_PROCESS_DURATION, absolute = true) @ResponseMetered(name = "rename-view", absolute = true) - public Response renameView(@PathParam("prefix") String prefix, RenameTableRequest request) { - icebergCatalogWrapperManager.getOps(prefix).renameView(request); + public Response renameView( + @PathParam("prefix") String prefix, RenameTableRequest renameViewRequest) { + String catalogName = IcebergRestUtils.getCatalogName(prefix); + LOG.info( + "Rename Iceberg view, catalog: {}, source: {}, destination: {}.", + catalogName, + renameViewRequest.source(), + renameViewRequest.destination()); + IcebergRequestContext context = new IcebergRequestContext(httpServletRequest(), catalogName); + viewOperationDispatcher.renameView(context, renameViewRequest); return IcebergRestUtils.noContent(); } + + // HTTP request is null in Jersey test, override with a mock request when testing. + @VisibleForTesting + HttpServletRequest httpServletRequest() { + return httpRequest; + } } diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableEvent.java index 46b8514b8cb..b7ebf35b15f 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableEvent.java @@ -29,8 +29,8 @@ @DeveloperApi public class IcebergCreateTableEvent extends IcebergTableEvent { - private CreateTableRequest createTableRequest; - private LoadTableResponse loadTableResponse; + private final CreateTableRequest createTableRequest; + private final LoadTableResponse loadTableResponse; public IcebergCreateTableEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableFailureEvent.java index 07b033d3d3b..1d20a553ace 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableFailureEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTableFailureEvent.java @@ -27,7 +27,7 @@ /** Represent a failure event when creating Iceberg table failed. */ @DeveloperApi public class IcebergCreateTableFailureEvent extends IcebergTableFailureEvent { - private CreateTableRequest createTableRequest; + private final CreateTableRequest createTableRequest; public IcebergCreateTableFailureEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTablePreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTablePreEvent.java index 552f0d45249..927f11c5c79 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTablePreEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateTablePreEvent.java @@ -26,7 +26,7 @@ /** Represent a pre event before creating Iceberg table. */ @DeveloperApi public class IcebergCreateTablePreEvent extends IcebergTablePreEvent { - private CreateTableRequest createTableRequest; + private final CreateTableRequest createTableRequest; public IcebergCreateTablePreEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewEvent.java new file mode 100644 index 00000000000..ff9f306c209 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewEvent.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.requests.CreateViewRequest; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +/** Represent an event after creating Iceberg view successfully. */ +@DeveloperApi +public class IcebergCreateViewEvent extends IcebergViewEvent { + + private final CreateViewRequest createViewRequest; + private final LoadViewResponse loadViewResponse; + + public IcebergCreateViewEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + CreateViewRequest createViewRequest, + LoadViewResponse loadViewResponse) { + super(icebergRequestContext, viewIdentifier); + this.createViewRequest = + IcebergRestUtils.cloneIcebergRESTObject(createViewRequest, CreateViewRequest.class); + this.loadViewResponse = + IcebergRestUtils.cloneIcebergRESTObject(loadViewResponse, LoadViewResponse.class); + } + + public CreateViewRequest createViewRequest() { + return createViewRequest; + } + + public LoadViewResponse loadViewResponse() { + return loadViewResponse; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewFailureEvent.java new file mode 100644 index 00000000000..4da73ed18ba --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewFailureEvent.java @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.requests.CreateViewRequest; + +/** Represent a failure event when creating Iceberg view failed. */ +@DeveloperApi +public class IcebergCreateViewFailureEvent extends IcebergViewFailureEvent { + private final CreateViewRequest createViewRequest; + + public IcebergCreateViewFailureEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + CreateViewRequest createViewRequest, + Exception e) { + super(icebergRequestContext, viewIdentifier, e); + this.createViewRequest = + IcebergRestUtils.cloneIcebergRESTObject(createViewRequest, CreateViewRequest.class); + } + + public CreateViewRequest createViewRequest() { + return createViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewPreEvent.java new file mode 100644 index 00000000000..00eea5db73d --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergCreateViewPreEvent.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.iceberg.rest.requests.CreateViewRequest; + +/** Represent a pre event before creating Iceberg view. */ +@DeveloperApi +public class IcebergCreateViewPreEvent extends IcebergViewPreEvent { + private final CreateViewRequest createViewRequest; + + public IcebergCreateViewPreEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + CreateViewRequest createViewRequest) { + super(icebergRequestContext, viewIdentifier); + this.createViewRequest = createViewRequest; + } + + public CreateViewRequest createViewRequest() { + return createViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableEvent.java index 69f517676e4..23fe799c912 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableEvent.java @@ -25,7 +25,7 @@ /** Represent an event after dropping Iceberg table successfully. */ @DeveloperApi public class IcebergDropTableEvent extends IcebergTableEvent { - private boolean purgeRequested; + private final boolean purgeRequested; public IcebergDropTableEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableFailureEvent.java index 8c58bc08f1a..b93daf5cba0 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableFailureEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTableFailureEvent.java @@ -25,7 +25,7 @@ /** Represent a failure event when dropping Iceberg table failed. */ @DeveloperApi public class IcebergDropTableFailureEvent extends IcebergTableFailureEvent { - private boolean purgeRequested; + private final boolean purgeRequested; public IcebergDropTableFailureEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTablePreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTablePreEvent.java index b8301229a6a..357a43a3902 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTablePreEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropTablePreEvent.java @@ -25,7 +25,7 @@ /** Represent a pre event before dropping Iceberg table. */ @DeveloperApi public class IcebergDropTablePreEvent extends IcebergTablePreEvent { - private boolean purgeRequested; + private final boolean purgeRequested; public IcebergDropTablePreEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewEvent.java new file mode 100644 index 00000000000..7a17ca1f472 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewEvent.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent an event after dropping Iceberg view successfully. */ +@DeveloperApi +public class IcebergDropViewEvent extends IcebergViewEvent { + + public IcebergDropViewEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewFailureEvent.java new file mode 100644 index 00000000000..99502160b44 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewFailureEvent.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a failure event when dropping Iceberg view failed. */ +@DeveloperApi +public class IcebergDropViewFailureEvent extends IcebergViewFailureEvent { + + public IcebergDropViewFailureEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier, Exception e) { + super(icebergRequestContext, viewIdentifier, e); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewPreEvent.java new file mode 100644 index 00000000000..163a51e9b60 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergDropViewPreEvent.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a pre event before dropping Iceberg view. */ +@DeveloperApi +public class IcebergDropViewPreEvent extends IcebergViewPreEvent { + + public IcebergDropViewPreEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewEvent.java new file mode 100644 index 00000000000..6bc7d83ac64 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewEvent.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** + * Represent an event after listing Iceberg view successfully. + * + *

To optimize memory usage and avoid the potential overhead associated with storing a large + * number of views directly within the ListViewEvent, the actual views listed are not maintained in + * this event. + */ +@DeveloperApi +public class IcebergListViewEvent extends IcebergViewEvent { + public IcebergListViewEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewFailureEvent.java new file mode 100644 index 00000000000..400e6b43039 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewFailureEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a failure event when listing Iceberg view failed. */ +@DeveloperApi +public class IcebergListViewFailureEvent extends IcebergViewFailureEvent { + public IcebergListViewFailureEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier, Exception e) { + super(icebergRequestContext, viewIdentifier, e); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewPreEvent.java new file mode 100644 index 00000000000..f0bebca38b0 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergListViewPreEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a pre event before listing Iceberg view. */ +@DeveloperApi +public class IcebergListViewPreEvent extends IcebergViewPreEvent { + public IcebergListViewPreEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadTableEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadTableEvent.java index 16e5734c707..066250cb359 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadTableEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadTableEvent.java @@ -27,7 +27,7 @@ /** Represent an event after loading Iceberg table successfully. */ @DeveloperApi public class IcebergLoadTableEvent extends IcebergTableEvent { - private LoadTableResponse loadTableResponse; + private final LoadTableResponse loadTableResponse; public IcebergLoadTableEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewEvent.java new file mode 100644 index 00000000000..5f22fe1b62e --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewEvent.java @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +/** Represent an event after loading Iceberg view successfully. */ +@DeveloperApi +public class IcebergLoadViewEvent extends IcebergViewEvent { + private final LoadViewResponse loadViewResponse; + + public IcebergLoadViewEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + LoadViewResponse loadViewResponse) { + super(icebergRequestContext, viewIdentifier); + this.loadViewResponse = + IcebergRestUtils.cloneIcebergRESTObject(loadViewResponse, LoadViewResponse.class); + } + + public LoadViewResponse loadViewResponse() { + return loadViewResponse; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewFailureEvent.java new file mode 100644 index 00000000000..493c9a3c1e7 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewFailureEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a failure event when loading Iceberg view failed. */ +@DeveloperApi +public class IcebergLoadViewFailureEvent extends IcebergViewFailureEvent { + public IcebergLoadViewFailureEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier, Exception e) { + super(icebergRequestContext, viewIdentifier, e); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewPreEvent.java new file mode 100644 index 00000000000..5970edc69d6 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergLoadViewPreEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a pre event before loading Iceberg view. */ +@DeveloperApi +public class IcebergLoadViewPreEvent extends IcebergViewPreEvent { + public IcebergLoadViewPreEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableEvent.java index f26b79714f0..42eea5cb2c7 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableEvent.java @@ -27,7 +27,7 @@ /** Represent an event after rename Iceberg table successfully. */ @DeveloperApi public class IcebergRenameTableEvent extends IcebergTableEvent { - private RenameTableRequest renameTableRequest; + private final RenameTableRequest renameTableRequest; public IcebergRenameTableEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableFailureEvent.java index f64a85baccb..9525891b34a 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableFailureEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTableFailureEvent.java @@ -26,7 +26,7 @@ /** Represent an event when rename Iceberg table failed. */ @DeveloperApi public class IcebergRenameTableFailureEvent extends IcebergTableFailureEvent { - private RenameTableRequest renameTableRequest; + private final RenameTableRequest renameTableRequest; public IcebergRenameTableFailureEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTablePreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTablePreEvent.java index 579f23182f8..1808a1d3c9b 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTablePreEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameTablePreEvent.java @@ -26,7 +26,7 @@ /** Represent an pre event before rename an Iceberg table. */ @DeveloperApi public class IcebergRenameTablePreEvent extends IcebergTablePreEvent { - private RenameTableRequest renameTableRequest; + private final RenameTableRequest renameTableRequest; public IcebergRenameTablePreEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewEvent.java new file mode 100644 index 00000000000..985a35592bb --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewEvent.java @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.requests.RenameTableRequest; + +/** Represent an event after rename Iceberg view successfully. */ +@DeveloperApi +public class IcebergRenameViewEvent extends IcebergViewEvent { + private final RenameTableRequest renameViewRequest; + + public IcebergRenameViewEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + RenameTableRequest renameViewRequest) { + super(icebergRequestContext, viewIdentifier); + this.renameViewRequest = + IcebergRestUtils.cloneIcebergRESTObject(renameViewRequest, RenameTableRequest.class); + } + + public RenameTableRequest renameViewRequest() { + return renameViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewFailureEvent.java new file mode 100644 index 00000000000..f5b943fabc9 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewFailureEvent.java @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.iceberg.rest.requests.RenameTableRequest; + +/** Represent an event when rename Iceberg view failed. */ +@DeveloperApi +public class IcebergRenameViewFailureEvent extends IcebergViewFailureEvent { + private final RenameTableRequest renameViewRequest; + + public IcebergRenameViewFailureEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + RenameTableRequest renameViewRequest, + Exception e) { + super(icebergRequestContext, viewIdentifier, e); + this.renameViewRequest = renameViewRequest; + } + + public RenameTableRequest renameViewRequest() { + return renameViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewPreEvent.java new file mode 100644 index 00000000000..7802b853429 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergRenameViewPreEvent.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.iceberg.rest.requests.RenameTableRequest; + +/** Represent an pre event before rename an Iceberg view. */ +@DeveloperApi +public class IcebergRenameViewPreEvent extends IcebergViewPreEvent { + private final RenameTableRequest renameViewRequest; + + public IcebergRenameViewPreEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + RenameTableRequest renameViewRequest) { + super(icebergRequestContext, viewIdentifier); + this.renameViewRequest = renameViewRequest; + } + + public RenameTableRequest renameViewRequest() { + return renameViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewEvent.java new file mode 100644 index 00000000000..9f3f1677eee --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewEvent.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.requests.UpdateTableRequest; +import org.apache.iceberg.rest.responses.LoadViewResponse; + +/** Represent an event after updating Iceberg view successfully. */ +@DeveloperApi +public class IcebergReplaceViewEvent extends IcebergViewEvent { + + private final UpdateTableRequest replaceViewRequest; + private final LoadViewResponse loadViewResponse; + + public IcebergReplaceViewEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest, + LoadViewResponse loadViewResponse) { + super(icebergRequestContext, viewIdentifier); + this.replaceViewRequest = + IcebergRestUtils.cloneIcebergRESTObject(replaceViewRequest, UpdateTableRequest.class); + this.loadViewResponse = + IcebergRestUtils.cloneIcebergRESTObject(loadViewResponse, LoadViewResponse.class); + } + + public UpdateTableRequest renameViewRequest() { + return replaceViewRequest; + } + + public LoadViewResponse loadViewResponse() { + return loadViewResponse; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewFailureEvent.java new file mode 100644 index 00000000000..b23e028aadd --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewFailureEvent.java @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.gravitino.iceberg.service.IcebergRestUtils; +import org.apache.iceberg.rest.requests.UpdateTableRequest; + +/** Represent a failure event when updating Iceberg view failed. */ +@DeveloperApi +public class IcebergReplaceViewFailureEvent extends IcebergViewFailureEvent { + private final UpdateTableRequest replaceViewRequest; + + public IcebergReplaceViewFailureEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest, + Exception e) { + super(icebergRequestContext, viewIdentifier, e); + this.replaceViewRequest = + IcebergRestUtils.cloneIcebergRESTObject(replaceViewRequest, UpdateTableRequest.class); + } + + public UpdateTableRequest replaceViewRequest() { + return replaceViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewPreEvent.java new file mode 100644 index 00000000000..aa15593aba6 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergReplaceViewPreEvent.java @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; +import org.apache.iceberg.rest.requests.UpdateTableRequest; + +/** Represent a pre event before updating Iceberg view. */ +@DeveloperApi +public class IcebergReplaceViewPreEvent extends IcebergViewPreEvent { + private final UpdateTableRequest replaceViewRequest; + + public IcebergReplaceViewPreEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + UpdateTableRequest replaceViewRequest) { + super(icebergRequestContext, viewIdentifier); + this.replaceViewRequest = replaceViewRequest; + } + + public UpdateTableRequest replaceViewRequest() { + return replaceViewRequest; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableExistsEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableExistsEvent.java index 4e54945f230..a8a8f3c1137 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableExistsEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableExistsEvent.java @@ -25,7 +25,7 @@ /** Represent an event after check Iceberg table exists successfully. */ @DeveloperApi public class IcebergTableExistsEvent extends IcebergTableEvent { - private boolean isExists; + private final boolean isExists; public IcebergTableExistsEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableFailureEvent.java index 4e2187f1a99..28c3fe08806 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableFailureEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergTableFailureEvent.java @@ -24,7 +24,7 @@ /** Represent a failure event when do Iceberg table operation failed. */ @DeveloperApi -public class IcebergTableFailureEvent extends IcebergFailureEvent { +public abstract class IcebergTableFailureEvent extends IcebergFailureEvent { protected IcebergTableFailureEvent( IcebergRequestContext icebergRequestContext, NameIdentifier nameIdentifier, Exception e) { super(icebergRequestContext, nameIdentifier, e); diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableEvent.java index 7e675213a2c..aebd27d6348 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableEvent.java @@ -29,8 +29,8 @@ @DeveloperApi public class IcebergUpdateTableEvent extends IcebergTableEvent { - private UpdateTableRequest updateTableRequest; - private LoadTableResponse loadTableResponse; + private final UpdateTableRequest updateTableRequest; + private final LoadTableResponse loadTableResponse; public IcebergUpdateTableEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableFailureEvent.java index 7624e248eef..0e288db04af 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableFailureEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTableFailureEvent.java @@ -27,7 +27,7 @@ /** Represent a failure event when updating Iceberg table failed. */ @DeveloperApi public class IcebergUpdateTableFailureEvent extends IcebergTableFailureEvent { - private UpdateTableRequest updateTableRequest; + private final UpdateTableRequest updateTableRequest; public IcebergUpdateTableFailureEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTablePreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTablePreEvent.java index a877a72fc2b..8c6ec378694 100644 --- a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTablePreEvent.java +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergUpdateTablePreEvent.java @@ -26,7 +26,7 @@ /** Represent a pre event before updating Iceberg table. */ @DeveloperApi public class IcebergUpdateTablePreEvent extends IcebergTablePreEvent { - private UpdateTableRequest updateTableRequest; + private final UpdateTableRequest updateTableRequest; public IcebergUpdateTablePreEvent( IcebergRequestContext icebergRequestContext, diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewEvent.java new file mode 100644 index 00000000000..2fd3c735d3a --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewEvent.java @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; + +/** Represents an abstract view post event in Gravitino Iceberg REST server. */ +public abstract class IcebergViewEvent extends IcebergEvent { + protected IcebergViewEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsEvent.java new file mode 100644 index 00000000000..ece8523208c --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsEvent.java @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent an event after check Iceberg view exists successfully. */ +@DeveloperApi +public class IcebergViewExistsEvent extends IcebergViewEvent { + private final boolean isExists; + + public IcebergViewExistsEvent( + IcebergRequestContext icebergRequestContext, + NameIdentifier viewIdentifier, + boolean isExists) { + super(icebergRequestContext, viewIdentifier); + this.isExists = isExists; + } + + public boolean isExists() { + return isExists; + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsFailureEvent.java new file mode 100644 index 00000000000..93aa839d400 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsFailureEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a failure event when check Iceberg view exist failed. */ +@DeveloperApi +public class IcebergViewExistsFailureEvent extends IcebergViewFailureEvent { + public IcebergViewExistsFailureEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier, Exception e) { + super(icebergRequestContext, viewIdentifier, e); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsPreEvent.java new file mode 100644 index 00000000000..938a7587ecb --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewExistsPreEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a pre event before checking Iceberg view exists. */ +@DeveloperApi +public class IcebergViewExistsPreEvent extends IcebergViewPreEvent { + public IcebergViewExistsPreEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier) { + super(icebergRequestContext, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewFailureEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewFailureEvent.java new file mode 100644 index 00000000000..4122bc32334 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewFailureEvent.java @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represent a failure event when do Iceberg view operation failed. */ +@DeveloperApi +public abstract class IcebergViewFailureEvent extends IcebergFailureEvent { + protected IcebergViewFailureEvent( + IcebergRequestContext icebergRequestContext, NameIdentifier viewIdentifier, Exception e) { + super(icebergRequestContext, viewIdentifier, e); + } +} diff --git a/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewPreEvent.java b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewPreEvent.java new file mode 100644 index 00000000000..2deaf49c370 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/main/java/org/apache/gravitino/listener/api/event/IcebergViewPreEvent.java @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.listener.api.event; + +import org.apache.gravitino.NameIdentifier; +import org.apache.gravitino.annotation.DeveloperApi; + +/** Represents an abstract view pre event in Gravitino Iceberg REST server. */ +@DeveloperApi +public abstract class IcebergViewPreEvent extends IcebergPreEvent { + protected IcebergViewPreEvent(IcebergRequestContext context, NameIdentifier viewIdentifier) { + super(context, viewIdentifier); + } +} diff --git a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/IcebergRestTestUtil.java b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/IcebergRestTestUtil.java index a2ee3d888ab..56b7fa2b023 100644 --- a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/IcebergRestTestUtil.java +++ b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/IcebergRestTestUtil.java @@ -39,6 +39,9 @@ import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableEventDispatcher; import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableOperationDispatcher; import org.apache.gravitino.iceberg.service.dispatcher.IcebergTableOperationExecutor; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewEventDispatcher; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationExecutor; import org.apache.gravitino.iceberg.service.extension.DummyCredentialProvider; import org.apache.gravitino.iceberg.service.metrics.IcebergMetricsManager; import org.apache.gravitino.iceberg.service.provider.IcebergConfigProvider; @@ -113,6 +116,11 @@ public static ResourceConfig getIcebergResourceConfig( IcebergTableEventDispatcher icebergTableEventDispatcher = new IcebergTableEventDispatcher( icebergTableOperationExecutor, eventBus, configProvider.getMetalakeName()); + IcebergViewOperationExecutor icebergViewOperationExecutor = + new IcebergViewOperationExecutor(icebergCatalogWrapperManager); + IcebergViewEventDispatcher icebergViewEventDispatcher = + new IcebergViewEventDispatcher( + icebergViewOperationExecutor, eventBus, configProvider.getMetalakeName()); IcebergMetricsManager icebergMetricsManager = new IcebergMetricsManager(new IcebergConfig()); resourceConfig.register( @@ -122,6 +130,7 @@ protected void configure() { bind(icebergCatalogWrapperManager).to(IcebergCatalogWrapperManager.class).ranked(2); bind(icebergMetricsManager).to(IcebergMetricsManager.class).ranked(2); bind(icebergTableEventDispatcher).to(IcebergTableOperationDispatcher.class).ranked(2); + bind(icebergViewEventDispatcher).to(IcebergViewOperationDispatcher.class).ranked(2); } }); } diff --git a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewOperations.java b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewOperations.java new file mode 100644 index 00000000000..ed229a0424f --- /dev/null +++ b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewOperations.java @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.iceberg.service.rest; + +import javax.inject.Inject; +import javax.servlet.http.HttpServletRequest; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; + +public class MockIcebergViewOperations extends IcebergViewOperations { + + @Inject + public MockIcebergViewOperations(IcebergViewOperationDispatcher viewOperationDispatcher) { + super(viewOperationDispatcher); + } + + // HTTP request is null in Jersey test, create a mock request + @Override + HttpServletRequest httpServletRequest() { + return IcebergRestTestUtil.createMockHttpRequest(); + } +} diff --git a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewRenameOperations.java b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewRenameOperations.java new file mode 100644 index 00000000000..694db5e2012 --- /dev/null +++ b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/MockIcebergViewRenameOperations.java @@ -0,0 +1,37 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.apache.gravitino.iceberg.service.rest; + +import javax.inject.Inject; +import javax.servlet.http.HttpServletRequest; +import org.apache.gravitino.iceberg.service.dispatcher.IcebergViewOperationDispatcher; + +public class MockIcebergViewRenameOperations extends IcebergViewRenameOperations { + @Inject + public MockIcebergViewRenameOperations(IcebergViewOperationDispatcher viewOperationDispatcher) { + super(viewOperationDispatcher); + } + + // HTTP request is null in Jersey test, create a mock request + @Override + HttpServletRequest httpServletRequest() { + return IcebergRestTestUtil.createMockHttpRequest(); + } +} diff --git a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergTableOperations.java b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergTableOperations.java index 59822d4bf0b..e6a6b393bab 100644 --- a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergTableOperations.java +++ b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergTableOperations.java @@ -210,7 +210,7 @@ void testTableExits() { verifyCreateTableSucc("exists_foo1"); dummyEventListener.clearEvent(); - verifyTableExistsStatusCode("exists_foo1", 200); + verifyTableExistsStatusCode("exists_foo1", 204); Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergTableExistsPreEvent); postEvent = dummyEventListener.popPostEvent(); Assertions.assertTrue(postEvent instanceof IcebergTableExistsEvent); @@ -430,7 +430,7 @@ private void verifyRenameTableSucc(String source, String dest) { Response response = doRenameTable(source, dest); System.out.println(response); System.out.flush(); - Assertions.assertEquals(Status.OK.getStatusCode(), response.getStatus()); + Assertions.assertEquals(Status.NO_CONTENT.getStatusCode(), response.getStatus()); } private void verifyRenameTableFail(String source, String dest, int status) { diff --git a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergViewOperations.java b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergViewOperations.java index 9ec2dc66f46..df896b1529b 100644 --- a/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergViewOperations.java +++ b/iceberg/iceberg-rest-server/src/test/java/org/apache/gravitino/iceberg/service/rest/TestIcebergViewOperations.java @@ -20,6 +20,7 @@ package org.apache.gravitino.iceberg.service.rest; import com.google.common.collect.ImmutableSet; +import java.util.Arrays; import java.util.Optional; import java.util.Set; import java.util.stream.Collectors; @@ -27,6 +28,27 @@ import javax.ws.rs.core.Application; import javax.ws.rs.core.MediaType; import javax.ws.rs.core.Response; +import org.apache.gravitino.listener.api.event.Event; +import org.apache.gravitino.listener.api.event.IcebergCreateViewEvent; +import org.apache.gravitino.listener.api.event.IcebergCreateViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergCreateViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergDropViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergListViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergLoadViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergRenameViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewFailureEvent; +import org.apache.gravitino.listener.api.event.IcebergReplaceViewPreEvent; +import org.apache.gravitino.listener.api.event.IcebergViewExistsEvent; +import org.apache.gravitino.listener.api.event.IcebergViewExistsPreEvent; import org.apache.iceberg.Schema; import org.apache.iceberg.UpdateRequirements; import org.apache.iceberg.catalog.Namespace; @@ -56,13 +78,17 @@ public class TestIcebergViewOperations extends TestIcebergNamespaceOperations { private static final String VIEW_QUERY = "select 1"; + private DummyEventListener dummyEventListener; + @Override protected Application configure() { + this.dummyEventListener = new DummyEventListener(); ResourceConfig resourceConfig = - IcebergRestTestUtil.getIcebergResourceConfig(IcebergViewOperations.class); + IcebergRestTestUtil.getIcebergResourceConfig( + MockIcebergViewOperations.class, true, Arrays.asList(dummyEventListener)); // create namespace before each view test resourceConfig.register(IcebergNamespaceOperations.class); - resourceConfig.register(IcebergViewRenameOperations.class); + resourceConfig.register(MockIcebergViewRenameOperations.class); return resourceConfig; } @@ -72,20 +98,31 @@ protected Application configure() { void testListViews(String prefix) { setUrlPathWithPrefix(prefix); verifyListViewFail(404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergListViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergListViewFailureEvent); verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyCreateViewSucc("list_foo1"); verifyCreateViewSucc("list_foo2"); + + dummyEventListener.clearEvent(); verifyLisViewSucc(ImmutableSet.of("list_foo1", "list_foo2")); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergListViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergListViewEvent); } @Test void testCreateView() { verifyCreateViewFail("create_foo1", 404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergCreateViewPreEvent); + Assertions.assertTrue( + dummyEventListener.popPostEvent() instanceof IcebergCreateViewFailureEvent); verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyCreateViewSucc("create_foo1"); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergCreateViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergCreateViewEvent); verifyCreateViewFail("create_foo1", 409); verifyCreateViewFail("", 400); @@ -94,10 +131,16 @@ void testCreateView() { @Test void testLoadView() { verifyLoadViewFail("load_foo1", 404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergLoadViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergLoadViewFailureEvent); verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyCreateViewSucc("load_foo1"); + + dummyEventListener.clearEvent(); verifyLoadViewSucc("load_foo1"); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergLoadViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergLoadViewEvent); verifyLoadViewFail("load_foo2", 404); } @@ -107,10 +150,19 @@ void testReplaceView() { verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyCreateViewSucc("replace_foo1"); ViewMetadata metadata = getViewMeta("replace_foo1"); + + dummyEventListener.clearEvent(); verifyReplaceSucc("replace_foo1", metadata); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergReplaceViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergReplaceViewEvent); verifyDropViewSucc("replace_foo1"); + + dummyEventListener.clearEvent(); verifyUpdateViewFail("replace_foo1", 404, metadata); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergReplaceViewPreEvent); + Assertions.assertTrue( + dummyEventListener.popPostEvent() instanceof IcebergReplaceViewFailureEvent); verifyDropNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyUpdateViewFail("replace_foo1", 404, metadata); @@ -119,22 +171,41 @@ void testReplaceView() { @Test void testDropView() { verifyDropViewFail("drop_foo1", 404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergDropViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergDropViewFailureEvent); + verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyDropViewFail("drop_foo1", 404); verifyCreateViewSucc("drop_foo1"); + + dummyEventListener.clearEvent(); verifyDropViewSucc("drop_foo1"); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergDropViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergDropViewEvent); + verifyLoadViewFail("drop_foo1", 404); } @Test void testViewExits() { verifyViewExistsStatusCode("exists_foo2", 404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergViewExistsPreEvent); + Event postEvent = dummyEventListener.popPostEvent(); + Assertions.assertTrue(postEvent instanceof IcebergViewExistsEvent); + Assertions.assertEquals(false, ((IcebergViewExistsEvent) postEvent).isExists()); + verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyViewExistsStatusCode("exists_foo2", 404); verifyCreateViewSucc("exists_foo1"); + dummyEventListener.clearEvent(); verifyViewExistsStatusCode("exists_foo1", 204); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergViewExistsPreEvent); + postEvent = dummyEventListener.popPostEvent(); + Assertions.assertTrue(postEvent instanceof IcebergViewExistsEvent); + Assertions.assertEquals(true, ((IcebergViewExistsEvent) postEvent).isExists()); + verifyLoadViewSucc("exists_foo1"); } @@ -144,11 +215,19 @@ void testRenameTable(String prefix) { setUrlPathWithPrefix(prefix); // namespace not exits verifyRenameViewFail("rename_foo1", "rename_foo3", 404); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergRenameViewPreEvent); + Assertions.assertTrue( + dummyEventListener.popPostEvent() instanceof IcebergRenameViewFailureEvent); verifyCreateNamespaceSucc(IcebergRestTestUtil.TEST_NAMESPACE_NAME); verifyCreateViewSucc("rename_foo1"); + + dummyEventListener.clearEvent(); // rename verifyRenameViewSucc("rename_foo1", "rename_foo2"); + Assertions.assertTrue(dummyEventListener.popPreEvent() instanceof IcebergRenameViewPreEvent); + Assertions.assertTrue(dummyEventListener.popPostEvent() instanceof IcebergRenameViewEvent); + verifyLoadViewFail("rename_foo1", 404); verifyLoadViewSucc("rename_foo2"); diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.sql b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.sql new file mode 100644 index 00000000000..7901ea41bc8 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.sql @@ -0,0 +1,57 @@ +CREATE SCHEMA gt_hive.gt_decimal_db1; + +USE gt_hive.gt_decimal_db1; + +CREATE TABLE test_decimal_bounds (amount DECIMAL(10, 2)); + +INSERT INTO test_decimal_bounds VALUES (12345.67), (-9999999.99), (0.01); + +INSERT INTO test_decimal_bounds VALUES (123456789.00); -- Exceeds precision + +SELECT * FROM test_decimal_bounds; + +CREATE TABLE test_decimal_aggregation (value DECIMAL(12, 3)); + +INSERT INTO test_decimal_aggregation VALUES (1234.567), (8901.234), (567.890); + +SELECT SUM(value) FROM test_decimal_aggregation; + +SELECT AVG(value) FROM test_decimal_aggregation; + +CREATE TABLE test_decimal_arithmetic (val1 DECIMAL(5, 2), val2 DECIMAL(4, 1)); + +INSERT INTO test_decimal_arithmetic VALUES (123.45,10.1); + +SELECT val1 + val2 FROM test_decimal_arithmetic; + +SELECT val1 * val2 FROM test_decimal_arithmetic; + +SELECT val1 / val2 FROM test_decimal_arithmetic; + +CREATE TABLE test_decimal_max_min (max_min_val DECIMAL(18, 4)); + +INSERT INTO test_decimal_max_min VALUES (99999999999999.9999); + +INSERT INTO test_decimal_max_min VALUES (-99999999999999.9999); + +INSERT INTO test_decimal_max_min VALUES (100000000000000.0000); -- Exceeds max + +SELECT * FROM test_decimal_max_min ORDER BY max_min_val; + +CREATE TABLE test_decimal_nulls (nullable_val DECIMAL(8, 2)); + +INSERT INTO test_decimal_nulls VALUES (NULL), (123.45), (NULL); + +SELECT * FROM test_decimal_nulls; + +DROP TABLE gt_hive.gt_decimal_db1.test_decimal_bounds; + +DROP TABLE gt_hive.gt_decimal_db1.test_decimal_aggregation; + +DROP TABLE gt_hive.gt_decimal_db1.test_decimal_arithmetic; + +DROP TABLE gt_hive.gt_decimal_db1.test_decimal_max_min; + +DROP TABLE gt_hive.gt_decimal_db1.test_decimal_nulls; + +DROP SCHEMA gt_hive.gt_decimal_db1; \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.txt b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.txt new file mode 100644 index 00000000000..6cecc4ab220 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00008_decimal.txt @@ -0,0 +1,62 @@ +CREATE SCHEMA + +USE + +CREATE TABLE + +INSERT: 3 rows + + Cannot cast DECIMAL(11, 2) '123456789.00' to DECIMAL(10, 2) + +"12345.67" +"-9999999.99" +"0.01" + +CREATE TABLE + +INSERT: 3 rows + +"10703.691" + +"3567.897" + +CREATE TABLE + +INSERT: 1 row + +"133.55" + +"1246.845" + +"12.22" + +CREATE TABLE + +INSERT: 1 row + +INSERT: 1 row + + Cannot cast DECIMAL(19, 4) '100000000000000.0000' to DECIMAL(18, 4) + +"-99999999999999.9999" +"99999999999999.9999" + +CREATE TABLE + +INSERT: 3 rows + +"" +"123.45" +"" + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP SCHEMA \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.sql b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.sql new file mode 100644 index 00000000000..77a60ea2d51 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.sql @@ -0,0 +1,65 @@ +CREATE SCHEMA gt_hive.gt_array_db1; + +USE gt_hive.gt_array_db1; + +CREATE TABLE test_array_basic (int_array ARRAY(INTEGER)); + +INSERT INTO test_array_basic VALUES (ARRAY[1, 2, 3]), (ARRAY[4, 5, NULL, 7]), (ARRAY[]); + +SELECT * FROM test_array_basic; + +SELECT int_array, CARDINALITY(int_array) AS array_length FROM test_array_basic; + +CREATE TABLE test_array_access (elements ARRAY(VARCHAR)); + +INSERT INTO test_array_access VALUES (ARRAY['apple', 'banana', 'cherry']); + +SELECT elements[1] AS first_element, elements[2] AS second_element FROM test_array_access; + +SELECT * FROM test_array_basic WHERE contains(int_array, 2); + +CREATE TABLE test_array_concat (array1 ARRAY(INTEGER), array2 ARRAY(INTEGER)); + +INSERT INTO test_array_concat VALUES (ARRAY[1, 2, 3], ARRAY[4, 5]); + +SELECT array1, array2, CONCAT(array1, array2) AS concatenated_array FROM test_array_concat; + +CREATE TABLE test_array_sort (unsorted_array ARRAY(INTEGER)); + +INSERT INTO test_array_sort VALUES (ARRAY[3, 1, 2]), (ARRAY[9, 7, 8]); + +SELECT unsorted_array, array_sort(unsorted_array) AS sorted_array FROM test_array_sort; + +CREATE TABLE test_array_nulls (mixed_array ARRAY(INTEGER)); + +INSERT INTO test_array_nulls VALUES (ARRAY[1, NULL, 3]), (ARRAY[NULL, NULL]); + +SELECT mixed_array, CARDINALITY(mixed_array) FROM test_array_nulls; + +CREATE TABLE test_array_agg (val INTEGER); + +INSERT INTO test_array_agg VALUES (1), (2), (3), (4); + +SELECT ARRAY_AGG(val) AS aggregated_array FROM test_array_agg; + +CREATE TABLE test_nested_array (nested_array ARRAY(ARRAY(VARCHAR))); + +INSERT INTO test_nested_array VALUES (ARRAY[ARRAY['a', 'b'], ARRAY['c', 'd']]); + +SELECT nested_array FROM test_nested_array; + +DROP TABLE gt_hive.gt_array_db1.test_array_basic; + +DROP TABLE gt_hive.gt_array_db1.test_array_access; + +DROP TABLE gt_hive.gt_array_db1.test_array_concat; + +DROP TABLE gt_hive.gt_array_db1.test_array_sort; + +DROP TABLE gt_hive.gt_array_db1.test_array_nulls; + +DROP TABLE gt_hive.gt_array_db1.test_array_agg; + +DROP TABLE gt_hive.gt_array_db1.test_nested_array; + +DROP SCHEMA gt_hive.gt_array_db1; \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.txt b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.txt new file mode 100644 index 00000000000..27c422586c1 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00009_array.txt @@ -0,0 +1,71 @@ +CREATE SCHEMA + +USE + +CREATE TABLE + +INSERT: 3 rows + +"[1, 2, 3]" +"[4, 5, NULL, 7]" +"[]" + +"[1, 2, 3]","3" +"[4, 5, NULL, 7]","4" +"[]","0" + +CREATE TABLE + +INSERT: 1 row + +"apple","banana" + +"[1, 2, 3]" + +CREATE TABLE + +INSERT: 1 row + +"[1, 2, 3]","[4, 5]","[1, 2, 3, 4, 5]" + +CREATE TABLE + +INSERT: 2 rows + +"[3, 1, 2]","[1, 2, 3]" +"[9, 7, 8]","[7, 8, 9]" + +CREATE TABLE + +INSERT: 2 rows + +"[1, NULL, 3]","3" +"[NULL, NULL]","2" + +CREATE TABLE + +INSERT: 4 rows + +"[1, 2, 3, 4]" + +CREATE TABLE + +INSERT: 1 row + +"[[a, b], [c, d]]" + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP SCHEMA \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.sql b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.sql new file mode 100644 index 00000000000..bbd89b92bc8 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.sql @@ -0,0 +1,49 @@ +CREATE SCHEMA gt_hive.gt_map_db1; + +USE gt_hive.gt_map_db1; + +CREATE TABLE test_map_nulls (string_map MAP(VARCHAR, VARCHAR)); + +INSERT INTO test_map_nulls VALUES (MAP(ARRAY['key1'], ARRAY[NULL])); + +INSERT INTO test_map_nulls VALUES (MAP(ARRAY[NULL], ARRAY['value1'])); + +SELECT * FROM test_map_nulls; + +INSERT INTO test_map_nulls VALUES (MAP(ARRAY[], ARRAY[])); + +SELECT * FROM test_map_nulls ORDER BY cardinality(string_map); + +INSERT INTO test_map_nulls VALUES (MAP(ARRAY['dup', 'dup'], ARRAY['value1', 'value2'])); + +CREATE TABLE test_map_types (int_decimal_map MAP(INTEGER, DECIMAL(10, 2))); + +INSERT INTO test_map_types VALUES (MAP(ARRAY[1, 2147483647], ARRAY[12345.67, 99999.99])); + +SELECT * FROM test_map_types; + +INSERT INTO test_map_nulls VALUES (MAP(ARRAY['k1', 'k2', 'k3'], ARRAY['v1', 'v2', 'v3'])); + +SELECT element_at(string_map, 'k1') AS key1_value, element_at(string_map, 'k3') AS key3_value FROM test_map_nulls ORDER BY key1_value; + +CREATE TABLE test_map_complex (map_of_arrays MAP(VARCHAR, ARRAY(INTEGER))); + +INSERT INTO test_map_complex VALUES (MAP(ARRAY['a', 'b'], ARRAY[ARRAY[1, 2], ARRAY[3, 4, 5]])); + +SELECT * FROM test_map_complex; + +CREATE TABLE test_map_aggregation (map_data MAP(VARCHAR, INTEGER)); + +INSERT INTO test_map_aggregation VALUES (MAP(ARRAY['a', 'b'], ARRAY[1, 2])), (MAP(ARRAY['a', 'b'], ARRAY[3, 4])); + +SELECT map_data['a'] AS key_a, SUM(map_data['b']) AS sum_b FROM test_map_aggregation GROUP BY map_data['a'] ORDER BY key_a; + +DROP TABLE gt_hive.gt_map_db1.test_map_nulls; + +DROP TABLE gt_hive.gt_map_db1.test_map_types; + +DROP TABLE gt_hive.gt_map_db1.test_map_complex; + +DROP TABLE gt_hive.gt_map_db1.test_map_aggregation; + +DROP SCHEMA gt_hive.gt_map_db1; \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.txt b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.txt new file mode 100644 index 00000000000..929dacfa1a5 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00010_map.txt @@ -0,0 +1,53 @@ +CREATE SCHEMA + +USE + +CREATE TABLE + +INSERT: 1 row + + map key cannot be null + +"{key1=NULL}" + +INSERT: 1 row + +"{}" +"{key1=NULL}" + + Duplicate map keys (dup) are not allowed + +CREATE TABLE + +INSERT: 1 row + +"{2147483647=99999.99, 1=12345.67}" + +INSERT: 1 row + +"v1","v3" +"","" +"","" + +CREATE TABLE + +INSERT: 1 row + +"{a=[1, 2], b=[3, 4, 5]}" + +CREATE TABLE + +INSERT: 2 rows + +"1","2" +"3","4" + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP SCHEMA \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.sql b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.sql new file mode 100644 index 00000000000..14c49edae7b --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.sql @@ -0,0 +1,91 @@ +CREATE SCHEMA gt_hive.gt_row_db1; + +USE gt_hive.gt_row_db1; + +CREATE TABLE test_row_basic (person ROW(id INTEGER, name VARCHAR)); + +CREATE TABLE source_tb1 (id INTEGER, name VARCHAR); + +INSERT INTO source_tb1 VALUES (1, 'Alice'), (2, NULL); + +INSERT INTO test_row_basic SELECT ROW(id, name) FROM source_tb1; + +SELECT * FROM test_row_basic ORDER BY person.id; + +INSERT INTO source_tb1 VALUES (3, 'Bob'); + +INSERT INTO test_row_basic SELECT ROW(id, name) FROM source_tb1; + +SELECT person.id AS person_id, person.name AS person_name FROM test_row_basic ORDER BY person.id; + +CREATE TABLE test_nested_row ( + person ROW(id INTEGER, name VARCHAR, address ROW(street VARCHAR, city VARCHAR)) +); + +CREATE TABLE source_tb2 (id INTEGER, name VARCHAR, street VARCHAR, city VARCHAR); + +INSERT INTO source_tb2 VALUES (1, 'Alice', '123 Elm St', 'Springfield'); + +INSERT INTO test_nested_row SELECT ROW(id, name, ROW(street, city)) FROM source_tb2; + +SELECT person.address.city AS city FROM test_nested_row; + +CREATE TABLE test_mixed_row ( + data ROW(int_val INTEGER, str_val VARCHAR, arr_val ARRAY(INTEGER), map_val MAP(VARCHAR, INTEGER)) +); + +CREATE TABLE source_tb3 (int_val INTEGER, str_val VARCHAR, arr_val ARRAY(INTEGER), map_val MAP(VARCHAR, INTEGER)); + +INSERT INTO source_tb3 VALUES (100, 'text', ARRAY[1, 2, 3], MAP(ARRAY['a', 'b'], ARRAY[10, 20])); + +INSERT INTO test_mixed_row SELECT ROW(int_val, str_val, arr_val, map_val) FROM source_tb3; + +SELECT * FROM test_mixed_row; + +INSERT INTO source_tb1 VALUES (NULL, NULL); + +INSERT INTO test_row_basic SELECT ROW(id, name) FROM source_tb1; + +SELECT * FROM test_row_basic ORDER BY person.id; + +CREATE TABLE test_row_in_array_map ( + row_array ARRAY(ROW(id INTEGER, name VARCHAR)), + row_map MAP(VARCHAR, ROW(age INTEGER, city VARCHAR)) +); + +CREATE TABLE source_tb5 (id INTEGER, name VARCHAR, age INTEGER, city VARCHAR); + +INSERT INTO source_tb5 VALUES (1, 'Alice', 30, 'NY'), (2, 'Bob', 40, 'LA'); + +INSERT INTO test_row_in_array_map SELECT ARRAY[ROW(id, name)], MAP(ARRAY['person1'], ARRAY[ROW(age, city)]) FROM source_tb5; + +INSERT INTO test_row_in_array_map +SELECT ARRAY_AGG(ROW(id, name)), MAP(ARRAY_AGG(person_key), ARRAY_AGG(ROW(age, city))) +FROM ( + SELECT id, name, age, city, CONCAT('person', CAST(ROW_NUMBER() OVER() AS VARCHAR)) AS person_key + FROM source_tb5 +) subquery; + +INSERT INTO source_tb1 VALUES (1, 'Alice'), (1, 'Alice'), (2, 'Bob'); + +INSERT INTO test_row_basic SELECT ROW(id, name) FROM source_tb1; + +SELECT person.id, COUNT(*) FROM test_row_basic GROUP BY person.id ORDER BY person.id; + +DROP TABLE gt_hive.gt_row_db1.test_row_basic; + +DROP TABLE gt_hive.gt_row_db1.source_tb1; + +DROP TABLE gt_hive.gt_row_db1.test_nested_row; + +DROP TABLE gt_hive.gt_row_db1.test_mixed_row; + +DROP TABLE gt_hive.gt_row_db1.source_tb2; + +DROP TABLE gt_hive.gt_row_db1.source_tb3; + +DROP TABLE gt_hive.gt_row_db1.test_row_in_array_map; + +DROP TABLE gt_hive.gt_row_db1.source_tb5; + +DROP SCHEMA gt_hive.gt_row_db1; \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.txt b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.txt new file mode 100644 index 00000000000..2183ef4387d --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00011_row.txt @@ -0,0 +1,95 @@ +CREATE SCHEMA + +USE + +CREATE TABLE + +CREATE TABLE + +INSERT: 2 rows + +INSERT: 2 rows + +"{id=1, name=Alice}" +"{id=2, name=NULL}" + +INSERT: 1 row + +INSERT: 3 rows + +"1","Alice" +"1","Alice" +"2","" +"2","" +"3","Bob" + +CREATE TABLE + +CREATE TABLE + +INSERT: 1 row + +INSERT: 1 row + +"Springfield" + +CREATE TABLE + +CREATE TABLE + +INSERT: 1 row + +INSERT: 1 row + +"{int_val=100, str_val=text, arr_val=[1, 2, 3], map_val={a=10, b=20}}" + +INSERT: 1 row + +INSERT: 4 rows + +"{id=1, name=Alice}" +"{id=1, name=Alice}" +"{id=1, name=Alice}" +"{id=2, name=NULL}" +"{id=2, name=NULL}" +"{id=2, name=NULL}" +"{id=3, name=Bob}" +"{id=3, name=Bob}" +"{id=NULL, name=NULL}" + +CREATE TABLE + +CREATE TABLE + +INSERT: 2 rows + +INSERT: 2 rows + +INSERT: 1 row + +INSERT: 3 rows + +INSERT: 7 rows + +"1","6" +"2","5" +"3","3" +"","2" + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP SCHEMA \ No newline at end of file diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.sql b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.sql new file mode 100644 index 00000000000..d36a4b2caf4 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.sql @@ -0,0 +1,176 @@ +CREATE SCHEMA gt_hive.gt_format_db1; + +USE gt_hive.gt_format_db1; + +CREATE TABLE storage_formats_orc ( +c_boolean boolean, +c_tinyint tinyint, +c_smallint smallint, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='ORC'); + +INSERT INTO storage_formats_orc +VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino')); + +SELECT * FROM storage_formats_orc; + +CREATE TABLE storage_formats_textfile ( +c_boolean boolean, +c_tinyint tinyint, +c_smallint smallint, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='TextFile'); + +INSERT INTO storage_formats_textfile +VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino')); + +SELECT * FROM storage_formats_textfile; + +CREATE TABLE storage_formats_parquet ( +c_boolean boolean, +c_tinyint tinyint, +c_smallint smallint, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='PARQUET'); + +INSERT INTO storage_formats_parquet VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino')); + +SELECT * FROM storage_formats_parquet; + +CREATE TABLE storage_formats_rcfile ( +c_boolean boolean, +c_tinyint tinyint, +c_smallint smallint, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='RCFile'); + +INSERT INTO storage_formats_rcfile VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino')); + +SELECT * FROM storage_formats_rcfile; + +CREATE TABLE storage_formats_sequencefile ( +c_boolean boolean, +c_tinyint tinyint, +c_smallint smallint, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='SequenceFile'); + +INSERT INTO storage_formats_sequencefile VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino') +); + +SELECT * FROM storage_formats_sequencefile; + +CREATE TABLE storage_formats_avro ( +c_boolean boolean, +c_tinyint integer, +c_smallint integer, +c_int integer, +c_bigint bigint, +c_real real, +c_double double, +c_decimal_10_0 decimal(10,0), +c_decimal_10_2 decimal(10,2), +c_decimal_38_5 decimal(38,5), +c_char char(10), +c_varchar varchar(10), +c_string varchar, +c_binary varbinary, +c_date date, +c_timestamp timestamp, +c_array array(integer), +c_map map(varchar, varchar), +c_row row(f1 integer, f2 varchar) +) WITH (format='AVRO'); + +INSERT INTO storage_formats_avro VALUES (true,127,32767,2147483647,9223372036854775807,123.345,234.567,346,12345678.91,1234567890123456789012.34567,'ala ma ','ala ma kot','ala ma kota',X'62696e61727920636f6e74656e74',DATE '2024-11-11',TIMESTAMP '2024-11-11 12:15:35.123',ARRAY[1, 2, 3],MAP(ARRAY['foo'], ARRAY['bar']),ROW(42, 'Trino')); + +SELECT * FROM storage_formats_avro; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_orc; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_textfile; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_parquet; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_rcfile; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_sequencefile; + +DROP TABLE gt_hive.gt_format_db1.storage_formats_avro; + +DROP SCHEMA gt_hive.gt_format_db1; diff --git a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.txt b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.txt new file mode 100644 index 00000000000..95a31519f80 --- /dev/null +++ b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/hive/00012_format.txt @@ -0,0 +1,53 @@ +CREATE SCHEMA + +USE + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +CREATE TABLE + +INSERT: 1 row + +"true","127","32767","2147483647","9223372036854775807","123.345","234.567","346","12345678.91","1234567890123456789012.34567","ala ma ","ala ma kot","ala ma kota","62 69 6e 61 72 79 20 63 6f 6e 74 65 6e 74","2024-11-11","2024-11-11 12:15:35.123","[1, 2, 3]","{foo=bar}","{f1=42, f2=Trino}" + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP TABLE + +DROP SCHEMA \ No newline at end of file diff --git a/web/web/src/app/metalakes/metalake/rightContent/CreateTableDialog.js b/web/web/src/app/metalakes/metalake/rightContent/CreateTableDialog.js index 1132964b494..df0091a2684 100644 --- a/web/web/src/app/metalakes/metalake/rightContent/CreateTableDialog.js +++ b/web/web/src/app/metalakes/metalake/rightContent/CreateTableDialog.js @@ -85,7 +85,7 @@ import { groupBy } from 'lodash-es' import { genUpdates } from '@/lib/utils' import { nameRegex, nameRegexDesc, keyRegex } from '@/lib/utils/regex' import { useSearchParams } from 'next/navigation' -import { relationalTypes } from '@/lib/utils/initial' +import { tableColumnTypes } from '@/lib/utils/initial' // Default form values const defaultFormValues = { @@ -142,6 +142,7 @@ const CreateTableDialog = props => { const [innerProps, setInnerProps] = useState([]) const [tableColumns, setTableColumns] = useState([{ name: '', type: '', nullable: true, comment: '' }]) const [initialTableData, setInitialTableData] = useState() + const [selectedColumnIndex, setSelectedColumnIndex] = useState(null) const dispatch = useAppDispatch() // Initialize form with react-hook-form @@ -205,6 +206,34 @@ const CreateTableDialog = props => { } } + // reset type suffix and param errors + if (field === 'type') { + updatedColumns[index].typeSuffix = '' + updatedColumns[index].paramErrors = '' + if (tableColumnTypes.find(type => type.key === value)?.params) { + updatedColumns[index].paramValues = [] + } + } + + setTableColumns(updatedColumns) + setValue('columns', updatedColumns) + } + + const transformParamValues = index => { + let updatedColumns = [...tableColumns] + + const validateParams = tableColumnTypes.find(type => type.key === updatedColumns[index].type)?.validateParams + const paramValues = updatedColumns[index].paramValues.filter(param => param !== undefined).map(Number) + const validateResult = validateParams(paramValues) + + if (validateResult.valid) { + updatedColumns[index].typeSuffix = `(${paramValues.join(',')})` + updatedColumns[index].paramErrors = '' + } else { + updatedColumns[index].paramErrors = validateResult.message + } + + updatedColumns[index].paramValues = undefined setTableColumns(updatedColumns) setValue('columns', updatedColumns) } @@ -303,7 +332,9 @@ const CreateTableDialog = props => { filteredCols.findIndex(otherCol => otherCol !== col && otherCol.name.trim() === col.name.trim()) !== -1 ) - if (hasDuplicateKeys || hasInvalidKeys || hasDuplicateColumnNames) { + const hasInvalidColumnTypes = tableColumns.some(col => col.paramErrors) + + if (hasDuplicateKeys || hasInvalidKeys || hasDuplicateColumnNames || hasInvalidColumnTypes) { return } @@ -321,7 +352,14 @@ const CreateTableDialog = props => { const tableData = { name: formData.name, comment: formData.comment, - columns: formData.columns.map(({ hasDuplicateName, ...rest }) => rest), + + // remove redundant fields + columns: formData.columns.map(({ hasDuplicateName, paramErrors, typeSuffix, ...rest }) => { + return { + ...rest, + type: rest.type + typeSuffix || '' // combine type and type suffix, like decimal(10,2) + } + }), properties } @@ -381,6 +419,13 @@ const CreateTableDialog = props => { // Set uniqueId to the column name to detect changes column.uniqueId = column.name + // Extract type suffix for types with parameters + const match = column.type.match(/(\w+)(\([\d,]+\))/) + if (match && match.length === 3) { + column.typeSuffix = match[2] + column.type = match[1] + } + return { ...column } @@ -401,10 +446,32 @@ const CreateTableDialog = props => { } }, [open, data, setValue, type]) + // Handle click outside of table rows + useEffect(() => { + const handleClickOutside = e => { + const selectElements = document.querySelectorAll('[role="listbox"]') + const isClickInsideSelect = Array.from(selectElements).some(el => el.contains(e.target)) + if (isClickInsideSelect) { + return + } + + const isClickInsideTableCell = e.target.closest('td') + if (isClickInsideTableCell) { + return + } + + setSelectedColumnIndex(null) + } + + document.addEventListener('click', handleClickOutside) + + return () => document.removeEventListener('click', handleClickOutside) + }, []) + return (

{ - Name + Name Type Nullable - Comment + Comment Action @@ -512,25 +579,73 @@ const CreateTableDialog = props => { )} - + setSelectedColumnIndex(index)}> - - {!column.type.trim() && ( - Type is required - )} + + + + {!column.type.trim() && ( + Type is required + )} + {column.paramErrors && ( + + {column.paramErrors} + + )} + + {selectedColumnIndex === index && + column.type && + (() => { + // Process typeSuffix before mapping + if (column.typeSuffix && !column.paramValues) { + const paramStr = column.typeSuffix.slice(1, -1) // Remove parentheses + const values = paramStr.split(',').map(v => v.trim()) + handleColumnChange({ + index, + field: 'paramValues', + value: values + }) + } + + return tableColumnTypes + .find(t => t.key === column.type) + ?.params?.map((param, paramIndex) => ( + { + const newParamValues = [...(column.paramValues || [])] + newParamValues[paramIndex] = e.target.value + handleColumnChange({ index, field: 'paramValues', value: newParamValues }) + }} + placeholder={`${param}`} + data-refer={`column-param-${index}-${paramIndex}`} + inputProps={{ min: 0 }} + /> + )) + })()} + {selectedColumnIndex !== index && + tableColumnTypes.find(type => type.key === column.type)?.params && + column.paramValues && + transformParamValues(index)} + diff --git a/web/web/src/lib/utils/index.js b/web/web/src/lib/utils/index.js index d2dde2a4b9b..17444dfa80e 100644 --- a/web/web/src/lib/utils/index.js +++ b/web/web/src/lib/utils/index.js @@ -102,7 +102,7 @@ export const genUpdates = (originalData, newData) => { newFieldName: newColumnsMap[key].name }) } - if (originalColumnsMap[key].type !== newColumnsMap[key].type) { + if ((originalColumnsMap[key].type + originalColumnsMap[key].typeSuffix || '') !== newColumnsMap[key].type) { updates.push({ '@type': 'updateColumnType', fieldName: [newColumnsMap[key].name], diff --git a/web/web/src/lib/utils/initial.js b/web/web/src/lib/utils/initial.js index 9efd67a7a5c..1ca2854dadb 100644 --- a/web/web/src/lib/utils/initial.js +++ b/web/web/src/lib/utils/initial.js @@ -347,21 +347,127 @@ export const providers = [ } ] -export const relationalTypes = [ - { label: 'Boolean', value: 'boolean' }, - { label: 'Byte', value: 'byte' }, - { label: 'Short', value: 'short' }, - { label: 'Integer', value: 'integer' }, - { label: 'Long', value: 'long' }, - { label: 'Float', value: 'float' }, - { label: 'Double', value: 'double' }, - { label: 'Date', value: 'date' }, - { label: 'Time', value: 'time' }, - { label: 'Timestamp', value: 'timestamp' }, - { label: 'Timestamp_tz', value: 'timestamp_tz' }, - { label: 'String', value: 'string' }, - { label: 'Interval_day', value: 'interval_day' }, - { label: 'Interval_year', value: 'interval_year' }, - { label: 'Uuid', value: 'uuid' }, - { label: 'Binary', value: 'binary' } +export const tableColumnTypes = [ + { key: 'boolean' }, + { key: 'byte' }, + { key: 'short' }, + { key: 'integer' }, + { key: 'long' }, + { key: 'float' }, + { key: 'double' }, + { + key: 'decimal', + params: ['precision', 'scale'], + validateParams: params => { + if (params.length !== 2) { + return { + valid: false, + message: 'Please set precision and scale' + } + } + + const [param1, param2] = params + if (param1 <= 0 || param1 > 38) { + return { + valid: false, + message: 'The precision must be between 1 and 38' + } + } + + if (param2 < 0 || param2 > param1) { + return { + valid: false, + message: 'The scale must be between 0 and the precision' + } + } + + return { + valid: true + } + } + }, + { key: 'date' }, + { key: 'time' }, + { key: 'timestamp' }, + { key: 'timestamp_tz' }, + { key: 'string' }, + { + key: 'char', + params: ['length'], + validateParams: params => { + if (params.length !== 1) { + return { + valid: false, + message: 'Please set length' + } + } + + const length = params[0] + + if (length <= 0) { + return { + valid: false, + message: 'The length must be greater than 0' + } + } + + return { + valid: true + } + } + }, + { + key: 'varchar', + params: ['length'], + validateParams: params => { + if (params.length !== 1) { + return { + valid: false, + message: 'Please set length' + } + } + + const length = params[0] + + if (length <= 0) { + return { + valid: false, + message: 'The length must be greater than 0' + } + } + + return { + valid: true + } + } + }, + { key: 'interval_day' }, + { key: 'interval_year' }, + { + key: 'fixed', + params: ['length'], + validateParams: params => { + if (params.length !== 1) { + return { + valid: false, + message: 'Please set length' + } + } + + const length = params[0] + + if (length <= 0) { + return { + valid: false, + message: 'The length must be greater than 0' + } + } + + return { + valid: true + } + } + }, + { key: 'uuid' }, + { key: 'binary' } ]