Merge branch 'main' into apache#5550-add-partition-pre-event-for-Grav…

…itino-server

# Conflicts:
#	docs/gravitino-server-config.md

CONTRIBUTING.md

@@ -19,9 +19,9 @@
 
 # Contributing to Apache Gravitino
 
-Thank you for your interest in contributing to Apache Gravitino! You are welcome to contribute in any way you can to enhance the project. Gravitino appreciates your assistance in making it better, whether through code contributions, documentation, tests, best practices, graphic design, or any other means that have a positive impact.
+Thank you for your interest in contributing to Apache Gravitino! Your support is invaluable in enhancing the project, and we warmly welcome contributions of all kinds. Gravitino appreciates your assistance in making it better, whether through code contributions, documentation, tests, best practices, graphic design, or any other means that have a positive impact.
 
-Before you get started, please read and follow these guidelines to ensure a smooth and productive collaboration.
+To ensure a smooth and productive collaboration, please take a moment to review and follow our contribution guidelines before getting started.
 
 ## Table of contents
 
@@ -82,7 +82,7 @@ To stop and start a local Gravitino server via `bin/gravitino.sh start` and `bin
     which java
     ```
 
-IntelliJ IDEA is an integrated development environment (IDE) for Java development. Setting the project SDK ensures that IntelliJ uses the correct Java version for building and running the project.
+IntelliJ IDEA is an integrated development environment (IDE) for Java development. Setting the project SDK ensures that IntelliJ uses the correct Java version to build and run the project.
 
 You can open up WSL in the IntelliJ terminal. Find the down arrow and select ubuntu.
 
@@ -94,9 +94,9 @@ You can open up WSL in the IntelliJ terminal. Find the down arrow and select ubu
 
 1. Open VSCode extension marketplace, search for and install **WSL**.
 
-2. On Windows, press `Ctrl+Shift+P` to open the command palette, and run `Shell Command: Install 'code' command in PATH`.
+2. Press `Ctrl+Shift+P` to open the command palette, and run `Shell Command: Install 'code' command in PATH`.
 
-Installing the WSL extension in VSCode allows you to open and edit files in your WSL environment directly from VSCode. Adding the `code` command to your PATH enables you to open VSCode from the WSL terminal.
+Installing the WSL extension in VSCode allows you to open and edit files in your WSL environment directly from VSCode. Adding the `code` command to your PATH lets you open VSCode from the WSL terminal.
 
 #### Verify and Configure Environment Variables
 
@@ -116,7 +116,7 @@ Installing the WSL extension in VSCode allows you to open and edit files in your
     C:\Users\epic\AppData\Local\Programs\Microsoft VS Code\bin
     ```
 
-Adding VSCode to the environment variables ensures that you can open VSCode from any command prompt or terminal window.
+Adding VSCode to the environment variables ensures you can open VSCode from any command prompt or terminal window.
 
 **On Ubuntu (WSL):**
 
@@ -148,7 +148,7 @@ Running `code --version` verifies that the `code` command is available. Using `c
    code .
    ```
 
-  This command will open the current WSL directory in VSCode on Windows. If you haven't added `code` to your path, follow these steps:
+  This command will open the current WSL directory in VSCode. If you haven't added `code` to your path, follow these steps:
 
 - Open VSCode on Windows.
 
@@ -158,15 +158,15 @@ Running `code --version` verifies that the `code` command is available. Using `c
 
 3. **Ensure Remote - WSL is Active**
 
-   When VSCode opens, you should see a green bottom-left corner indicating that VSCode is connected to WSL. If it isn't, click on the green area and select `Remote-WSL: New Window` or `Remote-WSL: Reopen Folder in WSL`.
+   When VSCode opens, you should see a green bottom-left corner indicating that VSCode is connected to WSL. If not, click on the green area and select `Remote-WSL: New Window` or `Remote-WSL: Reopen Folder in WSL`.
 
 4. **Edit and Develop Your Project**
 
    You can now edit and develop your project files in VSCode as if they were local files. The Remote - WSL extension seamlessly bridges the file system between Windows and WSL.
 
 ### Handling Memory Issues in WSL
 
-If you ran into a memory issue when using WSL, here are some solutions to resolve it.
+Here are some solutions if you encounter a memory issue when using WSL.
 
 1. **Shut down WSL**  
    If your WSL is open, you can shut it down in Windows PowerShell using the following command:  
@@ -177,7 +177,7 @@ If you ran into a memory issue when using WSL, here are some solutions to resolv
 
 2. **Navigate to user folder**  
 
-   Open up File Explorer and navigate to `C:\Users\<your-username>`.
+   Open File Explorer and navigate to `C:\Users\<your-username>`.
 
 3. **Create the `.wslconfig` file**  
 
@@ -207,7 +207,7 @@ For project management policies, refer to [GOVERNANCE.md](GOVERNANCE.md).
 
 ### Future Development Directions
 
-For future development directions, refer to the [ROADMAP.md](ROADMAP.md) document.
+Refer to the [ROADMAP.md](ROADMAP.md) document for future development directions.
 
 ## Contribution guidelines
 
@@ -217,7 +217,7 @@ Please read and follow the [Code of Conduct](CODE_OF_CONDUCT.md). Gravitino prov
 
 ### Reporting bugs
 
-If you find a bug in Gravitino, please open an issue on GitHub. Be sure to include as much detail as possible, such as a clear description, steps to reproduce, and your environment. Please follow the template provided. If you encounter a security issue, please refer to [SECURITY.md](SECURITY.md).
+If you find a bug in Gravitino, please open an issue on GitHub. Include as much detail as possible, such as a clear description, reproduction steps, and your environment. Please follow the template provided. If you encounter a security issue, please refer to [SECURITY.md](SECURITY.md).
 
 ### Suggesting enhancements
 
@@ -259,32 +259,32 @@ For details on the review process, please refer to [MAINTAINERS.md](MAINTAINERS.
 
 ## Testing
 
-The CI infrastructure runs unit and integration tests on each pull request, please make sure these tests pass before making a pull request.
+The CI infrastructure runs unit and integration tests on each pull request. Please make sure these tests pass before making a pull request.
 
-The unit tests run on every build and integration tests run as needed. See [how to test](docs/how-to-test.md) for more information.
+The unit tests run on every build, and integration tests run as needed. See [how to test](docs/how-to-test.md) for more information.
 
-When adding new code or fixing a bug be sure to add unit tests to provide coverage.
+Add unit tests to provide coverage when adding new code or fixing a bug.
 
 ## Coding standards
 
-Spotless checks code formatting. If your code isn't correctly formatted, the build fails. To correctly format your code please use Spotless.
+Spotless checks code formatting. If your code is correctly formatted, the build succeeds. To format your code correctly, please use Spotless.
 
 ```bash
 ./gradlew spotlessApply
 ```
 
-All files must have a license header and the build fails if any files are missing license headers. If you are adding third-party code be sure to understand how to add the third-party license to Gravitino LICENSE and NOTICE files.
+All files must have a license header, and the build fails if any files are missing license headers. If you are adding third-party code, you may need to add the third-party license to Gravitino's LICENSE and NOTICE files. If you are unsure of how to do this, please ask for help.
 
-For any bugs or new code please add unit tests to provide coverage of the code. The project may not accept code without unit tests.
+Please add unit tests to provide code coverage for any bugs or new code. The project may not accept code without unit tests.
 
-All text files should use macOS/unix style line endings (LF) not windows style line endings (CRLF).
+All text files should use macOS/unix style line endings (LF), not Windows-style line endings (CRLF).
 
 ## Community and communication
 
-Join the [community mailing list](https://lists.apache.org/list.html?[email protected]) to discuss ideas and seek help and are also encouraged to use GitHub discussions.
+Join the [community mailing list](https://lists.apache.org/[email protected]) to discuss ideas and seek help. You are also encouraged to use GitHub discussions.
 
 ## License
 
-When contributing to this project, you agree that your contributions use the Apache License version 2. Please ensure you have permission to do this if required by your employer.
+When contributing to this project, you agree that your contributions use the Apache License version 2. Please ensure you have permission to do this if your employer requires it.
 
 Thank you for your contributions to Gravitino! The project appreciates your help in making it a success.

build.gradle.kts

@@ -771,6 +771,7 @@ tasks {
     subprojects.forEach() {
       if (!it.name.startsWith("catalog") &&
         !it.name.startsWith("authorization") &&
+        !it.name.startsWith("cli") &&
         !it.name.startsWith("client") && !it.name.startsWith("filesystem") && !it.name.startsWith("spark") && !it.name.startsWith("iceberg") && it.name != "trino-connector" &&
         it.name != "integration-test" && it.name != "bundled-catalog" && !it.name.startsWith("flink") &&
         it.name != "integration-test" && it.name != "hive-metastore-common" && !it.name.startsWith("flink") &&

clients/filesystem-hadoop3/src/main/java/org/apache/gravitino/filesystem/hadoop/GravitinoVirtualFileSystem.java

@@ -25,12 +25,14 @@
 import com.github.benmanes.caffeine.cache.Scheduler;
 import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
 import com.google.common.collect.Maps;
 import com.google.common.util.concurrent.ThreadFactoryBuilder;
 import java.io.File;
 import java.io.IOException;
 import java.net.URI;
 import java.util.Arrays;
+import java.util.List;
 import java.util.Map;
 import java.util.concurrent.ScheduledThreadPoolExecutor;
 import java.util.concurrent.ThreadFactory;
@@ -57,6 +59,8 @@
 import org.apache.hadoop.fs.FileSystem;
 import org.apache.hadoop.fs.Path;
 import org.apache.hadoop.fs.permission.FsPermission;
+import org.apache.hadoop.security.Credentials;
+import org.apache.hadoop.security.token.Token;
 import org.apache.hadoop.util.Progressable;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -559,6 +563,19 @@ public long getDefaultBlockSize(Path f) {
     return context.getFileSystem().getDefaultBlockSize(context.getActualFileLocation());
   }
 
+  @Override
+  public Token<?>[] addDelegationTokens(String renewer, Credentials credentials) {
+    List<Token<?>> tokenList = Lists.newArrayList();
+    for (FileSystem fileSystem : internalFileSystemCache.asMap().values()) {
+      try {
+        tokenList.addAll(Arrays.asList(fileSystem.addDelegationTokens(renewer, credentials)));
+      } catch (IOException e) {
+        Logger.warn("Failed to add delegation tokens for filesystem: {}", fileSystem.getUri(), e);
+      }
+    }
+    return tokenList.stream().distinct().toArray(Token[]::new);
+  }
+
   @Override
   public synchronized void close() throws IOException {
     // close all actual FileSystems

core/src/main/java/org/apache/gravitino/listener/CatalogEventDispatcher.java

@@ -33,14 +33,19 @@
 import org.apache.gravitino.exceptions.NonEmptyEntityException;
 import org.apache.gravitino.listener.api.event.AlterCatalogEvent;
 import org.apache.gravitino.listener.api.event.AlterCatalogFailureEvent;
+import org.apache.gravitino.listener.api.event.AlterCatalogPreEvent;
 import org.apache.gravitino.listener.api.event.CreateCatalogEvent;
 import org.apache.gravitino.listener.api.event.CreateCatalogFailureEvent;
+import org.apache.gravitino.listener.api.event.CreateCatalogPreEvent;
 import org.apache.gravitino.listener.api.event.DropCatalogEvent;
 import org.apache.gravitino.listener.api.event.DropCatalogFailureEvent;
+import org.apache.gravitino.listener.api.event.DropCatalogPreEvent;
 import org.apache.gravitino.listener.api.event.ListCatalogEvent;
 import org.apache.gravitino.listener.api.event.ListCatalogFailureEvent;
+import org.apache.gravitino.listener.api.event.ListCatalogPreEvent;
 import org.apache.gravitino.listener.api.event.LoadCatalogEvent;
 import org.apache.gravitino.listener.api.event.LoadCatalogFailureEvent;
+import org.apache.gravitino.listener.api.event.LoadCatalogPreEvent;
 import org.apache.gravitino.listener.api.info.CatalogInfo;
 import org.apache.gravitino.utils.PrincipalUtils;
 
@@ -68,6 +73,7 @@ public CatalogEventDispatcher(EventBus eventBus, CatalogDispatcher dispatcher) {
 
   @Override
   public NameIdentifier[] listCatalogs(Namespace namespace) throws NoSuchMetalakeException {
+    eventBus.dispatchEvent(new ListCatalogPreEvent(PrincipalUtils.getCurrentUserName(), namespace));
     try {
       NameIdentifier[] nameIdentifiers = dispatcher.listCatalogs(namespace);
       eventBus.dispatchEvent(new ListCatalogEvent(PrincipalUtils.getCurrentUserName(), namespace));
@@ -81,6 +87,7 @@ public NameIdentifier[] listCatalogs(Namespace namespace) throws NoSuchMetalakeE
 
   @Override
   public Catalog[] listCatalogsInfo(Namespace namespace) throws NoSuchMetalakeException {
+    eventBus.dispatchEvent(new ListCatalogPreEvent(PrincipalUtils.getCurrentUserName(), namespace));
     try {
       Catalog[] catalogs = dispatcher.listCatalogsInfo(namespace);
       eventBus.dispatchEvent(new ListCatalogEvent(PrincipalUtils.getCurrentUserName(), namespace));
@@ -94,6 +101,7 @@ public Catalog[] listCatalogsInfo(Namespace namespace) throws NoSuchMetalakeExce
 
   @Override
   public Catalog loadCatalog(NameIdentifier ident) throws NoSuchCatalogException {
+    eventBus.dispatchEvent(new LoadCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident));
     try {
       Catalog catalog = dispatcher.loadCatalog(ident);
       eventBus.dispatchEvent(
@@ -115,6 +123,10 @@ public Catalog createCatalog(
       String comment,
       Map<String, String> properties)
       throws NoSuchMetalakeException, CatalogAlreadyExistsException {
+    CatalogInfo catalogInfo =
+        new CatalogInfo(ident.name(), type, provider, comment, properties, null);
+    eventBus.dispatchEvent(
+        new CreateCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident, catalogInfo));
     try {
       Catalog catalog = dispatcher.createCatalog(ident, type, provider, comment, properties);
       eventBus.dispatchEvent(
@@ -134,6 +146,8 @@ public Catalog createCatalog(
   @Override
   public Catalog alterCatalog(NameIdentifier ident, CatalogChange... changes)
       throws NoSuchCatalogException, IllegalArgumentException {
+    eventBus.dispatchEvent(
+        new AlterCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident, changes));
     try {
       Catalog catalog = dispatcher.alterCatalog(ident, changes);
       eventBus.dispatchEvent(
@@ -150,6 +164,7 @@ public Catalog alterCatalog(NameIdentifier ident, CatalogChange... changes)
   @Override
   public boolean dropCatalog(NameIdentifier ident, boolean force)
       throws NonEmptyEntityException, CatalogInUseException {
+    eventBus.dispatchEvent(new DropCatalogPreEvent(PrincipalUtils.getCurrentUserName(), ident));
     try {
       boolean isExists = dispatcher.dropCatalog(ident, force);
       eventBus.dispatchEvent(

core/src/main/java/org/apache/gravitino/listener/api/event/AlterCatalogPreEvent.java

@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.CatalogChange;
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.annotation.DeveloperApi;
+
+/** Represents an event triggered before altering a catalog. */
+@DeveloperApi
+public final class AlterCatalogPreEvent extends CatalogPreEvent {
+  private final CatalogChange[] catalogChanges;
+
+  public AlterCatalogPreEvent(
+      String user, NameIdentifier identifier, CatalogChange[] catalogChanges) {
+    super(user, identifier);
+    this.catalogChanges = catalogChanges;
+  }
+
+  /**
+   * Retrieves the specific changes that were made to the catalog during the alteration process.
+   *
+   * @return An array of {@link CatalogChange} objects detailing each modification applied to the
+   *     catalog.
+   */
+  public CatalogChange[] catalogChanges() {
+    return catalogChanges;
+  }
+}

core/src/main/java/org/apache/gravitino/listener/api/event/CatalogPreEvent.java

@@ -0,0 +1,31 @@
+/*
+ *  Licensed to the Apache Software Foundation (ASF) under one
+ *  or more contributor license agreements.  See the NOTICE file
+ *  distributed with this work for additional information
+ *  regarding copyright ownership.  The ASF licenses this file
+ *  to you under the Apache License, Version 2.0 (the
+ *  "License"); you may not use this file except in compliance
+ *  with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing,
+ *  software distributed under the License is distributed on an
+ *  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ *  KIND, either express or implied.  See the License for the
+ *  specific language governing permissions and limitations
+ *  under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.annotation.DeveloperApi;
+
+/** Represents a pre-event for catalog operations. */
+@DeveloperApi
+public abstract class CatalogPreEvent extends PreEvent {
+  protected CatalogPreEvent(String user, NameIdentifier identifier) {
+    super(user, identifier);
+  }
+}

core/src/main/java/org/apache/gravitino/listener/api/event/CreateCatalogPreEvent.java

@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.gravitino.listener.api.event;
+
+import org.apache.gravitino.NameIdentifier;
+import org.apache.gravitino.annotation.DeveloperApi;
+import org.apache.gravitino.listener.api.info.CatalogInfo;
+
+/** Represents an event triggered before creating a catalog. */
+@DeveloperApi
+public class CreateCatalogPreEvent extends CatalogPreEvent {
+  private final CatalogInfo createCatalogRequest;
+
+  public CreateCatalogPreEvent(
+      String user, NameIdentifier identifier, CatalogInfo createCatalogRequest) {
+    super(user, identifier);
+    this.createCatalogRequest = createCatalogRequest;
+  }
+
+  /**
+   * Retrieves the create catalog request.
+   *
+   * @return A {@link CatalogInfo} instance encapsulating the comprehensive details of create
+   *     catalog request.
+   */
+  public CatalogInfo createCatalogRequest() {
+    return createCatalogRequest;
+  }
+}