Java 11 compatibility #5589
Comments
|
Just for reference, #4907 should be fixed first |
This was referenced Apr 13, 2019
This was referenced Apr 30, 2019
|
Now it is failed to launch druid with the tutorial under JDK 11: apache-druid-0.14.2-incubating$ bin/supervise -c quickstart/tutorial/conf/tutorial-cluster.conf
Please upgrade to Java 8 or better! Your current version is 11.0.2 |
|
Hi @itanoss, we don't support Java 11 yet. |
|
+1 on this issue, is there any ETA or anymore information about this topic ? Thanks a lot. |
This was referenced Aug 24, 2019
xvrl
added a commit
that referenced
this issue
Aug 25, 2019
* Fallback to parsing classpath for hadoop task in Java 9+ In Java 9 and above we cannot assume that the system classloader is an instance of URLClassLoader. This change adds a fallback method to parse the system classpath in that case, and adds a unit test to validate it matches what JDK8 would do. Note: This has not been tested in an actual hadoop setup, so this is mostly to help us pass unit tests. * Remove granularity test of dubious value One of our granularity tests relies on system classloader being a URLClassLoaders to catch a bug related to class initialization and static initializers using a subclass (see #2979) This test was added to catch a potential regression, but it assumes we would add back the same type of static initializers to this specific class, so it seems to be of dubious value as a unit test and mostly serves to illustrate the bug. relates to #5589
This was referenced Aug 25, 2019
|
FYI, the current state of this is that stuff appears to be basically working, minus Hadoop indexing as detailed in #8591. I am able to successfully run through the native batch 'quickstart' with java 11 and 12, query the data, and so on. Per this dev list thread I think we are going to defer claiming 'experimental' support for Java 11+ in a release until integration tests using Java 11 have been added. Currently all of the unit tests are being run with jdk11, so the adventurous among us might have some luck experimenting with Druid 0.16.0+ on newer versions of Java. |
|
An update - there are also issues with the DataSketches extension on Java 11, explained in #8647. Even though the tests pass on JDK 11 in master, the library doesn't officially support Java 11 and has some functionality that doesn't work right when run there. |
|
Adding integration tests for Java 11.
PR: #9249 |
|
Any news on this topic? It's been 3 years since the release of Java 11 with several improvements in GC and performance in general. Is there any plan or ETA for the support? :S |
|
There is experimental support for it, but not all extensions may work. It is unclear to me if DataSketches officially supports Java 11 yet or not. Our unit and integration tests all pas with Java 11, but I am not sure if there is more to it. There was some discussion as part of #8647 regarding the matter. All I can say is that we have been running in production with Java 11 and seen significant GC improvements for sure, but we don't exercise all the functionality, so there might be some gaps I am not aware of. It might be worthwhile to update the documentation and break down JDK support by Druid extension so that people can be more comfortable upgrading. cc @gianm @leerho |
|
A bunch of our environments run on Java 11 as well, and we use the kafka, kinesis, datasketches, and s3 extensions pretty heavily. Datasketches requires Java 8 to compile (https://github.com/apache/datasketches-java#jdk8hotspot-is-required-to-compile) and I'm not sure if it officially supports running on Java 11 or not. As a data point, though: we have been using HLL, theta, and quantiles sketches on Java 11 for a while now without any apparent problem. The biggest Druid feature that we don't use is any of the Hadoop integration. I'm not sure how well that works. So, a couple things:
It might be time to just say we support Java 11 in all cases, or if we can't resolve the above 2 questions, at least in some cases. |
|
@leerho Thanks for the update. It'll be exciting to have official datasketches support for the newer JDKs.
@tanisdlj It sounds like even though things aren't 100% sorted out yet, a few people (us included) are running on Java 11 in production without issue already, so you might have similar success if you just go for it. If you do try that: please let us know if you run into any issues. |
|
@gianm I will give it a try probably next week and let you know :D. |
|
AGH! no, actually we use DataSketches, so no java 11 for us Y_Y |
|
@tanisdlj well, we use datasketches too and Java 11 has been working fine for us. I guess that we must not be using the pieces of datasketches that aren't yet compatible with Java 11. So it could be worth a try, even though it's not officially supported yet. Fwiw, we are mostly using the hll, theta, and quantiles sketches. |
|
With the datasketches.java <= 2.0.0, and datasketches.memory <= 1.3.0, the
compatibility issue with JDK 9+ occurs when you try to create your own
off-heap (Direct) memory segments using the Memory component.
If you are using off-heap memory segments created with ByteBuffer and then
use datasketches to access them it should not be a problem.
…On Tue, Jun 8, 2021 at 11:07 AM Gian Merlino ***@***.***> wrote:
@tanisdlj <https://github.com/tanisdlj> well, we use datasketches too and
Java 11 has been working fine for us. I guess that we must not be using the
pieces of datasketches that aren't yet compatible with Java 11. So it could
be worth a try, even though it's not officially supported yet. Fwiw, we are
mostly using the hll, theta, and quantiles sketches.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5589 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADCXRQTY3O4HYYVRTZNEDJLTRZL4JANCNFSM4EZIPVXQ>
.
|
|
@gianm when I tried the overlord with Java 11 I am getting the following output: Java version Not sure if it's safe with those warning to push forward :/ |
|
@tanisdlj those warnings are harmless, they only indicate that this code will not be compatible with future JDK releases. You can disable the warning by adding the relevant |
|
Ok, tried Java 11 in Historicals and middlemanagers. |
|
Is there still an interest to support Java 11? |
|
or java 17? |
|
We've updated to a version of datasketches that officially supports JDK versions up through 13. In addition, we're also running integration tests on Java 11, and lots of people are running Java 11 in production. I think it's time to officially support JDK 11, at least. |
@tanisdlj Hmm, one idea about what's going on here. If you have not set a value for Another thing to check: is it possible that your two deployments have different values of |
|
Is anyone using Druid with Java 11 and Hadoop ingestion via the I'm asking since we're using Hadoop 2.x, which doesn't support compiling on JDK 11, but perhaps it supports running on it. |
|
Here's a patch: #12232 |
|
according to https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions 2.x doesn't support running on java 11, but it's possible the parts we use do. |
|
This is done. |
|
Next: #12838 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Java 11 is the next LTS for the JRE http://www.oracle.com/technetwork/java/javase/tech/eol-135779.html and is currently anticipated to be released sometime in September 2018. This ask is that, when available, Druid is compatible with running on the next long term supported java version.
The text was updated successfully, but these errors were encountered: