Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improvement] Compile fails when using JDK21 #906

Closed
justinmclean opened this issue Dec 1, 2023 · 13 comments · Fixed by #934
Closed

[Improvement] Compile fails when using JDK21 #906

justinmclean opened this issue Dec 1, 2023 · 13 comments · Fixed by #934
Assignees
@justinmclean @jerryshao
Labels
improvement Improvements on everything
Milestone
Gravitino 0.3.0

Comments

@justinmclean
Copy link
Member

What would you like to be improved?

Compile fails when using JDK21 with this error:
Build gravitino FAILURE reason:
Execution failed for task ':api:compileJava':
java.lang.RuntimeException: java.lang.NoSuchFieldError: Class com.sun.tools.javac.tree.JCTree$JCImport does not have member field 'com.sun.tools.javac.tree.JCTree qualid'

How should we improve?

Be able to compile when using JDK21
@justinmclean justinmclean added the improvement Improvements on everything label Dec 1, 2023
@yuqi1129
Copy link
Contributor

yuqi1129 commented Dec 1, 2023

It's OK on Ubuntu 22.04 with JDK21, maybe we should also test other Linux release OS including Ubuntu.

@justinmclean
Copy link
Member Author

justinmclean commented Dec 3, 2023 •
edited
Loading

I'm now getting different issues.

The current main fails on OSX due to a spotless issue:

  • What went wrong:
    Execution failed for task ':trino-connector:spotlessJavaCheck'.

The following files had format violations:
src/main/java/com/datastrato/gravitino/trino/connector/catalog/CatalogConnectorAdapter.java

uname -a
Darwin Justins-MBP.hub 22.6.0 Darwin Kernel Version 22.6.0: Fri Sep 15 13:41:28 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_ARM64_T6020 arm64
% java -version
openjdk version "21.0.1" 2023-10-17
OpenJDK Runtime Environment Homebrew (build 21.0.1)
OpenJDK 64-Bit Server VM Homebrew (build 21.0.1, mixed mode, sharing)

The current main fails on Ubuntu due to a spotless issue, but this looks different from the OSX one. It works on JDK8.

Task :integration-test:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/integration/test/MiniGravitinoContext.java':
null

uname -a
Linux ip-172-31-4-208 6.2.0-1012-aws #12~22.04.1-Ubuntu SMP Thu Sep  7 14:01:24 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
:~/gravitino$ java -version
openjdk version "21.0.1" 2023-10-17
OpenJDK Runtime Environment (build 21.0.1+12-Ubuntu-222.04)
OpenJDK 64-Bit Server VM (build 21.0.1+12-Ubuntu-222.04, mixed mode, sharing)

@justinmclean justinmclean self-assigned this Dec 4, 2023
@jerryshao
Copy link
Contributor

I don't have any issue testing on macOS with jdk21 as default java version. Please using the latest main branch and test again.

@jerryshao
Copy link
Contributor

Actually, our build enforces the compile jdk version to 8 and 17, no matter which version you installed in your OS.

@justinmclean justinmclean mentioned this issue Dec 4, 2023
Merged
@justinmclean
Copy link
Member Author

Steps to reproduce:

  1. find the correct path:
    /usr/libexec/java_home -V

  2. Set Java Home
    export JAVA_HOME="/opt/homebrew/Cellar/openjdk/21.0.1/libexec/openjdk.jdk/Contents/Home"

  3. test java version:

java -version 
openjdk version "21.0.1" 2023-10-17
OpenJDK Runtime Environment Homebrew (build 21.0.1)
OpenJDK 64-Bit Server VM Homebrew (build 21.0.1, mixed mode, sharing)
  1. Run ./gradlew build in gravitino. Error occurs:
> Task :integration-test:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/integration/test/MiniGravitino.java':

> Task :common:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/dto/util/DTOConverters.java':

> Task :clients:client-java:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/client/OAuth2ClientUtil.java':

> Task :catalogs:catalog-jdbc-common:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/catalog/jdbc/operation/JdbcDatabaseOperations.java':

> Task :core:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/metrics/source/MetricsSource.java':

@justinmclean
Copy link
Member Author

Given the issues are remove unused imports should be simple to fix - unless there are other issues.

@justinmclean
Copy link
Member Author

Also fails using JDK11 BTW

@jerryshao
Copy link
Contributor

Steps to reproduce:

  1. find the correct path:
    /usr/libexec/java_home -V
  2. Set Java Home
    export JAVA_HOME="/opt/homebrew/Cellar/openjdk/21.0.1/libexec/openjdk.jdk/Contents/Home"
  3. test java version:
java -version 
openjdk version "21.0.1" 2023-10-17
OpenJDK Runtime Environment Homebrew (build 21.0.1)
OpenJDK 64-Bit Server VM Homebrew (build 21.0.1, mixed mode, sharing)
  1. Run ./gradlew build in gravitino. Error occurs:
> Task :integration-test:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/integration/test/MiniGravitino.java':

> Task :common:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/dto/util/DTOConverters.java':

> Task :clients:client-java:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/client/OAuth2ClientUtil.java':

> Task :catalogs:catalog-jdbc-common:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/catalog/jdbc/operation/JdbcDatabaseOperations.java':

> Task :core:spotlessJava FAILED
Step 'removeUnusedImports' found problem in 'src/main/java/com/datastrato/gravitino/metrics/source/MetricsSource.java':

This actually is a spotless problem, not a gravitino problem, see diffplug/spotless#1819. We should upgrade the spotless version to fix this problem.

@jerryshao
Copy link
Contributor

Currently, we can have two choices:

  1. Fix the Gradle building environment to use JDK8 and declare that JDK8+ is not supported currently.
  2. Upgrade the Spotless plugin, which requires the building environment to use JDK11+ at least.

Both ways have pros and cons. I have a fix for solution 2, but it will introduce several JDK versions and confuse people.

@xunliu @qqqttt123 @yuqi1129 @FANNG1 @justinmclean what's your opinion?

@jerryshao jerryshao self-assigned this Dec 4, 2023
@jerryshao
Copy link
Contributor

JDK 21 is hard to support because it is hard to be compatible with JDK8 on java code style, so currently only support from 8 to 17.

@jerryshao
Copy link
Contributor

Also fails using JDK11 BTW

JDK11 should be worked, I verified several times, I guess you doesn't test in a clean environment, you still have Gradle deamon running when you switching to JDK11.

I see this problem only exists in JDK21.

@jerryshao jerryshao mentioned this issue Dec 4, 2023
Merged
@jerryshao jerryshao added this to the Gravitino 0.3.0 milestone Dec 4, 2023
@FANNG1
Copy link
Contributor

FANNG1 commented Dec 4, 2023

JDK 21 is hard to support because it is hard to be compatible with JDK8 on java code style, so currently only support from 8 to 17.

+1

@justinmclean
Copy link
Member Author

JDK11 and JDK17 are now working on the current main branch.

jerryshao added a commit that referenced this issue Dec 5, 2023
@jerryshao
[#906] fix(test): Fix spotless issue when running against JDK21 (#934)
f1e4e3d 
### What changes were proposed in this pull request?

Current spotless plugin has issues running against JDK21
(diffplug/spotless#1819), but we cannot
upgrade the spotless version to a newer version to fix this issue,
because newer version of Spotless requires at least JDK11. So for now,
we enforce the JDK version to <= 17 when we running Gradle.

### Why are the changes needed?

Fix the problem when Gradle running with JDK21.

Fix: #906 

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Verified locally on JDK 8, 11, 17 between macOS and Ubuntu.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@justinmclean justinmclean

@jerryshao jerryshao

Labels
improvement Improvements on everything
Projects
None yet
Milestone
Gravitino 0.3.0
Development

Successfully merging a pull request may close this issue.

[#906] fix(test): Fix spotless issue when running against JDK21 jerryshao/unified-catalog
4 participants
@justinmclean @jerryshao @FANNG1 @yuqi1129