Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AVRO-3716: Build under JDK 21 #2699

Merged
merged 6 commits into from
Feb 1, 2024
Merged

AVRO-3716: Build under JDK 21 #2699

merged 6 commits into from
Feb 1, 2024

Conversation

nielsbasjes
Copy link
Contributor

https://issues.apache.org/jira/browse/AVRO-3716

The base goal is to build Avro under the latest available JDK and plugins while maintaining Java 8 compatibility.

This changes a lot in the JAVA build ...

@github-actions github-actions bot added Java Pull Requests for Java binding build labels Jan 21, 2024
@nielsbasjes nielsbasjes changed the title Avro 3716 build under java21 AVRO-3716: Build under JDK 21 Jan 21, 2024
@nielsbasjes nielsbasjes force-pushed the AVRO-3716-BuildUnderJava21 branch from 980b421 to 89acee4 Compare January 21, 2024 13:20
@github-actions github-actions bot added the Rust label Jan 21, 2024
@nielsbasjes nielsbasjes force-pushed the AVRO-3716-BuildUnderJava21 branch from 59b3bca to 77f01fc Compare January 21, 2024 13:58
@github-actions github-actions bot removed the Rust label Jan 21, 2024
@nielsbasjes nielsbasjes force-pushed the AVRO-3716-BuildUnderJava21 branch 2 times, most recently from 0a152db to 9e50af5 Compare January 27, 2024 22:32
@nielsbasjes
Copy link
Contributor Author

Building the software using different JDK versions may yield different code than what is actually released; So the code is built only once under a specific JDK version and that is tested against compatibility with all other JDKs.

Summary of what I have changed:

  • The build requires Maven 3.9.6 and JDK 8, 11, 17 and 21 (yes, all of them) installed with the correct toolchains config to be present.
  • The build (Maven 3.9.6) must run under Java 21 so the latest plugins can be used.
  • Depending on the module the maximum JDK is selected to build the software with a release target of Java 8.
  • Using the invoker plugin all tests of the main avro module are rerun in all mentioned JDKs (yes, all tests).
  • The interoperability tests were restructured into a separate module which generates van validates the files.
  • An extra profile skipQuality which disables all testing and code quality verification.

Rough edges:

  • The code relating to the Unsafe had to be removed because building it under Java 9+ meant the need for specific compiler flags that disallowed targeting Java 8.
  • Some tests required Mockito which now need Java 11+.
  • Some tests rely on tools.jar which no longer exists in java 9+
  • Several tests relating to reflection fail on newer Java versions because it tried to access internal fields in different modules.

@nielsbasjes
Copy link
Contributor Author

The java build fails over a file copy.avro missing in a generated artifact that is the OUTPUT of a test. This is a bug that needs fixing.

@nielsbasjes nielsbasjes force-pushed the AVRO-3716-BuildUnderJava21 branch from 9e50af5 to 35a4664 Compare January 28, 2024 09:50
@nielsbasjes nielsbasjes self-assigned this Jan 28, 2024
@nielsbasjes nielsbasjes marked this pull request as ready for review January 28, 2024 13:42
opwvhk
opwvhk reviewed Jan 30, 2024
View reviewed changes
lang/java/avro/pom.xml Outdated Show resolved Hide resolved
opwvhk
opwvhk reviewed Jan 30, 2024
View reviewed changes
lang/java/avro/pom.xml Outdated Show resolved Hide resolved
opwvhk
opwvhk reviewed Jan 30, 2024
View reviewed changes
lang/java/ipc/pom.xml Outdated Show resolved Hide resolved
@iemejia
Copy link
Member

iemejia commented Jan 31, 2024 •
edited
Loading

Hi, I am busy until the second week of february so I won't be able to give any feedback on this. I will take a look afterwards in case the PR is still ongoing. (Sorry!)

dependabot bot added 3 commits February 1, 2024 11:28
@dependabot @nielsbasjes
Bump slf4j.version from 1.7.36 to 2.0.11 in /lang/java
0e29fb0 
Bumps `slf4j.version` from 1.7.36 to 2.0.11.

Updates `org.slf4j:slf4j-api` from 1.7.36 to 2.0.11

Updates `org.slf4j:slf4j-simple` from 1.7.36 to 2.0.11

Updates `org.slf4j:slf4j-log4j12` from 1.7.36 to 2.0.11

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-api
  dependency-type: direct:production
  update-type: version-update:semver-major
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:development
  update-type: version-update:semver-major
- dependency-name: org.slf4j:slf4j-log4j12
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @nielsbasjes
Bump org.mockito:mockito-core from 4.11.0 to 5.9.0 in /lang/java
efad56a 
Bumps [org.mockito:mockito-core](https://github.com/mockito/mockito) from 4.11.0 to 5.9.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](mockito/mockito@v4.11.0...v5.9.0)

---
updated-dependencies:
- dependency-name: org.mockito:mockito-core
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @nielsbasjes
Bump org.apache.thrift:libthrift from 0.16.0 to 0.19.0 in /lang/java
d764149 
Bumps [org.apache.thrift:libthrift](https://github.com/apache/thrift) from 0.16.0 to 0.19.0.
- [Release notes](https://github.com/apache/thrift/releases)
- [Changelog](https://github.com/apache/thrift/blob/master/CHANGES.md)
- [Commits](apache/thrift@v0.16.0...v0.19.0)

---
updated-dependencies:
- dependency-name: org.apache.thrift:libthrift
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
dependabot bot and others added 3 commits February 1, 2024 11:28
@dependabot @nielsbasjes
Bump com.diffplug.spotless:spotless-maven-plugin in /lang/java
3b5f949 
Bumps [com.diffplug.spotless:spotless-maven-plugin](https://github.com/diffplug/spotless) from 2.27.2 to 2.42.0.
- [Changelog](https://github.com/diffplug/spotless/blob/main/CHANGES.md)
- [Commits](diffplug/spotless@maven/2.27.2...lib/2.42.0)

---
updated-dependencies:
- dependency-name: com.diffplug.spotless:spotless-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot @nielsbasjes
Bump antlr.version from 4.9.3 to 4.13.1 in /lang/java
9c73911 
Bumps `antlr.version` from 4.9.3 to 4.13.1.

Updates `org.antlr:antlr4-runtime` from 4.9.3 to 4.13.1
- [Release notes](https://github.com/antlr/antlr4/releases)
- [Changelog](https://github.com/antlr/antlr4/blob/dev/CHANGES.txt)
- [Commits](antlr/antlr4@4.9.3...4.13.1)

Updates `org.antlr:antlr4-maven-plugin` from 4.9.3 to 4.13.1
- [Release notes](https://github.com/antlr/antlr4/releases)
- [Changelog](https://github.com/antlr/antlr4/blob/dev/CHANGES.txt)
- [Commits](antlr/antlr4@4.9.3...4.13.1)

---
updated-dependencies:
- dependency-name: org.antlr:antlr4-runtime
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.antlr:antlr4-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@nielsbasjes
AVRO-3716: Build under Java21
36a534a
@nielsbasjes nielsbasjes force-pushed the AVRO-3716-BuildUnderJava21 branch from 9f80509 to 36a534a Compare February 1, 2024 12:27
@nielsbasjes
Copy link
Contributor Author

I have squashed the commits. If the build passed I'm merging it.

@nielsbasjes nielsbasjes merged commit 9038f23 into apache:main Feb 1, 2024
64 of 65 checks passed
@nielsbasjes nielsbasjes deleted the AVRO-3716-BuildUnderJava21 branch February 1, 2024 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@martin-g martin-g martin-g left review comments

@opwvhk opwvhk opwvhk approved these changes

@iemejia iemejia Awaiting requested review from iemejia

@Fokko Fokko Awaiting requested review from Fokko

Assignees

@nielsbasjes nielsbasjes

Labels
build Java Pull Requests for Java binding
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nielsbasjes @iemejia @martin-g @opwvhk