Java 9 and Jigsaw compatibility #2571

jbduncan · 2016-09-21T19:36:01Z

I wasn't able to find an issue for this, so apologies if this has been raised already.

Has Guava been tested with any of the early-access Java 9 and/or Jigsaw releases yet?

If not, this seems to be a pretty important medium-to-long-term thing to look into, as Guava currently uses sun.misc.Unsafe in a number of places, which by my understanding would prevent Guava from being used in Jigsaw-enabled versions of Java, including Java 9.

kevinb9n · 2016-09-21T20:06:58Z

Guava's usage of Unsafe always has a fallback if any problem occurs.

On Wed, Sep 21, 2016 at 12:36 PM, Jonathan Bluett-Duncan <
[email protected]> wrote:

I wasn't able to find an issue for this, so apologies if this has been
raised already.

Has Guava been tested with any of the early-access Java 9 and/or Jigsaw
releases yet?

If not, this seems to be a pretty important medium-to-long-term thing to
look into, as Guava currently uses sun.misc.Unsafe in a number of places,
which by my understanding would prevent Guava from being used in
Jigsaw-enabled versions of Java, including Java 9

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2571, or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5Cl6sN5NjLotWT34upHBri7CQqC54Aks5qsYcpgaJpZM4KDMZj
.

Kevin Bourrillion | Java Librarian | Google, Inc. | [email protected]

jbduncan · 2016-09-21T20:49:49Z

Yes, that's true actually!

However, I think the fallbacks work on the assumption that sun.misc.Unsafe is a class which exists and which can be made accessible by reflection, which I believe has been "fixed" in Jigsaw by either removing sun.misc.Unsafe completely or moving it to a jdk-internal module which can only be accessed with a command-line argument. If I'm mistaken on this, then I'd be more than happy to be proved so. :)

But despite all this, I still think it's important to test Guava against Java 9 and Jigsaw if it's not been done so already, in case there are other crucial areas where backwards-compatibility has been broken and would otherwise prevent open source Guava's uptake by Java 9 users in the future.

kevinb9n · 2016-09-22T20:48:59Z

Ah, I see.

We'll be receptive to specific fix requests, but probably won't be able to put much thought into this ourselves until we get to the point of testing migrating ourselves to Java 9. (Bear in mind we finally successfully moved to Java 8 last week! But Java 9 should definitely be a quicker project.)

jbduncan · 2016-09-22T23:28:29Z

Ah yes, I learned about your successful migration to Java 8 the other day. Congratulations!

Going back to the subject of Java 9 though, you may very well be right about it being quicker to migrate to than Java 8.

On one hand, it'll help that with the release of MRJARs, one can have build artifacts that continue to use internal APIs like com.sun.* on Java 8 but use officially supported replacements on Java 9+. But on the other hand, if there are places in Guava where com.sun.* is vital and Oracle have not provided sufficient replacement APIs, then Guava may end up being stuck on Java 8, which concerns me.

I'm also concerned by how OpenJDK 9 has, at the time of writing of me writing this, gone past the Feature Complete milestone, which may make improving the replacement APIs difficult if needed.

What are your thoughts on this @kevinb9n?

Also, whereas I don't feel qualified enough to thoroughly test Guava against Java 9 & Jigsaw and to talk to Oracle in your stead (as I'm not familiar with all of Guava's intricacies), I am more than happy to run Guava against Java 9 and reply back to this issue if I discover anything. Would you be happy if I did this @kevinb9n?

octylFractal · 2016-09-24T03:51:45Z

According to jdeps -jdkinternals, Guava only relies on sun.misc.Unsafe, which has been mostly replaced by variable handles in Java 9. There might be some other things that are different but not explicitly depended on. I'm going to try making a fork of Guava that uses Java 9 as it's JDK requirement and run my small projects on that for testing.

jbduncan · 2016-09-24T21:14:04Z

I just ran jdeps -jdkinternals using jigsaw-jdk-9-ea+136_windows-x64_bin on my copy of Guava, and it seems that sun.security.jca.{ProviderList, Providers} and sun.misc.FpUtils are used in test code as well. But I'm not so concerned about that, because the tests which use those APIs are more-or-less one off cases, and it looks like they can be changed, disabled or removed if needed.

I also did a half-thorough search of the Guava non-test codebase, and it seems that @kevinb9n was actually right, and I was wrong!

Although using sun.misc.Unsafe on a JVM which doesn't have that class would cause exceptions/errors to be thrown by the Guava classes that use it, it seems the exceptions do get caught eventually, higher up all the potential call chains, by other Guava classes, and that viable alternatives would be used whenever they happen.

So it seems this issue is actually not as urgent as I first thought it was! 😝

octylFractal · 2016-09-24T21:26:09Z

I also just ran through master and replaced all the usages of Unsafe with VarHandle. The code looks much cleaner, and it was a quick & easy transition. So handling this when Java 9 is released probably shouldn't be too much of a hassle.

The main pain I encountered while doing this was that the tools used in the maven build can't handle Java 9 yet, so I can't actually make a build of it immediately.

jbduncan · 2016-09-24T21:36:41Z

@kenzierocks Thanks for looking into that.

So I suppose this would only be urgent then if Guava starts having users who try to compile the Guava sources or tests with JDK 9+ and they find that certain sun.* classes which Guava depends on have been removed. But we won't really know until Maven supports Java 9.

I also don't know enough to know exactly which classes have been planned for removal/module encapsulation in JDK 9 and which are just deprecated (jdeps doesn't seem to give me the whole picture here), so more research would be needed.

jbduncan · 2016-09-24T21:50:40Z

The transition might be made even easier by the lexicographical Arrays::compare methods, which could potentially be used to implement or replace UnsignedBytes.lexicographicalComparator.

chrisseaton · 2017-01-19T11:23:33Z

I found this use of an internal JDK field which doesn't work on Java 9.

b5b4b40#commitcomment-20536128

There's a fallback, but it just does nothing. Is that a problem? What goes wrong if we use the fallback?

ClassLoader that loaded FinalizableReferenceQueue from being garbage-collected subsequently. This is Guava issue http://code.google.com/p/guava-libraries/issues/detail?id=92 ------------- Created by MOE: http://code.google.com/p/moe-java MOE_MIGRATED_REVID=28624233

cpovirk · 2017-01-19T15:02:55Z

@eamonnmcmanus might know what we'd need to do.

eamonnmcmanus · 2017-01-23T18:40:26Z

@cpovirk can you log a separate issue for that and assign it to me?

cpovirk · 2017-01-23T19:06:25Z

Filed #2723

…n scheme: https://issues.apache.org/jira/browse/MJAVADOC-442 (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158429834

FpUtils is going away in Java 9. We'd use Math under Java 7, too, but it's not present until Java 8. (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158434703

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158475485

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158545119

(I got confused when I added an argLine in the parent pom (as part of Java 9 testing) and found it had no effect, apparently because it's overwritten in the child.) (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158738701

Apparently 2.8.40 is out, but currently we're internally upgrading to 2.7.19, so that seems like the safest thing to target. (But I'm sure we'll get to 2.8.40 or later eventually, so maybe I'm wasting time bothering with 2.7.19....) (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158745717

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158747678

Under previous JDKs, we continue to output "java.util.Map.java.util.Map.Entry," just as those JDKs do. (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=159237707

cowwoc · 2017-08-22T05:49:15Z

Filed #2920

eamonnmcmanus · 2017-10-04T16:34:50Z

(I'm already assigned on #2723 and I don't think I'm otherwise involved here.)

jbduncan · 2017-10-04T18:38:40Z

(SGTM, @eamonnmcmanus. Many thanks in advance!)

cpovirk · 2017-10-04T19:31:12Z

@netdpb did a bunch of work here after I stopped, and it's been released in 23.1.

Remaining:

Automatic-Module-Name missing from sub-module manifests #2920 (Automatic-Module-Name on submodules)
inheritableThreadLocals won't be accessible under Java 9 #2723 (inheritableThreadLocals)

I don't believe there's anything to do with Unsafe yet, as it's remaining exposed in Java 9.

Let's track remaining work with the separate bugs. Please open more if you encounter more problems.

(followup to CL 158434703 to make the Android tests runnable under Java 9) (part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=172804202

…and deprecate JAVA_EXT_DIRS. value()'s return type has been @nullable since 4232450, but I had forgotten the details about why, despite past CLs to update the test for specific missing keys. I assume that we'll never actually *remove* JAVA_EXT_DIRS. (Just slightly relevant to #2571) RELNOTES=`base`: Deprecated `StandardSystemProperty.JAVA_EXT_DIRS`. We do not plan to remove the API, but in recent versions of Java, that property always has a value of `null`. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=345251412

chrisseaton mentioned this issue Jan 19, 2017

Can't make private JDK fields accessible in Java 9 jnr/jnr-ffi#105

Closed

cpovirk mentioned this issue Jan 23, 2017

inheritableThreadLocals won't be accessible under Java 9 #2723

Closed

jbduncan mentioned this issue Apr 11, 2017

Remove jsr305 dependency #2782

Closed

kashike mentioned this issue Jun 7, 2017

Java 9 compatibility #2842

Closed

ansell mentioned this issue Jun 8, 2017

Changes for #15 to get closer to building/testing with Java-9 gbif/gbif-api#16

Merged

cpovirk added a commit that referenced this issue Jun 9, 2017

Skip static methods, like the forthcoming Set.of.

9121020

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158475485

cpovirk added a commit that referenced this issue Jun 9, 2017

Don't expect java.ext.dirs under Java 9.

16f42af

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158545119

cpovirk added a commit that referenced this issue Jun 12, 2017

Recognize Java-9-style version strings.

8311be9

(part of #2571) ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=158747678

eljobe mentioned this issue Jun 15, 2017

Add an Automatic-Module-Name manifest entry #2846

Closed

ronshapiro assigned cpovirk Oct 4, 2017

ronshapiro assigned eamonnmcmanus Oct 4, 2017

eamonnmcmanus removed their assignment Oct 4, 2017

cpovirk closed this as completed Oct 4, 2017

netdpb mentioned this issue Dec 3, 2020

MOE Sync 2020-12-03 #5343

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java 9 and Jigsaw compatibility #2571

Java 9 and Jigsaw compatibility #2571

jbduncan commented Sep 21, 2016 •

edited

Loading

kevinb9n commented Sep 21, 2016

jbduncan commented Sep 21, 2016 •

edited

Loading

kevinb9n commented Sep 22, 2016

jbduncan commented Sep 22, 2016

octylFractal commented Sep 24, 2016

jbduncan commented Sep 24, 2016 •

edited

Loading

octylFractal commented Sep 24, 2016

jbduncan commented Sep 24, 2016 •

edited

Loading

jbduncan commented Sep 24, 2016

chrisseaton commented Jan 19, 2017

cpovirk commented Jan 19, 2017

eamonnmcmanus commented Jan 23, 2017

cpovirk commented Jan 23, 2017

cowwoc commented Aug 22, 2017

eamonnmcmanus commented Oct 4, 2017

jbduncan commented Oct 4, 2017

cpovirk commented Oct 4, 2017

Java 9 and Jigsaw compatibility #2571

Java 9 and Jigsaw compatibility #2571

Comments

jbduncan commented Sep 21, 2016 • edited Loading

kevinb9n commented Sep 21, 2016

jbduncan commented Sep 21, 2016 • edited Loading

kevinb9n commented Sep 22, 2016

jbduncan commented Sep 22, 2016

octylFractal commented Sep 24, 2016

jbduncan commented Sep 24, 2016 • edited Loading

octylFractal commented Sep 24, 2016

jbduncan commented Sep 24, 2016 • edited Loading

jbduncan commented Sep 24, 2016

chrisseaton commented Jan 19, 2017

cpovirk commented Jan 19, 2017

eamonnmcmanus commented Jan 23, 2017

cpovirk commented Jan 23, 2017

cowwoc commented Aug 22, 2017

eamonnmcmanus commented Oct 4, 2017

jbduncan commented Oct 4, 2017

cpovirk commented Oct 4, 2017

jbduncan commented Sep 21, 2016 •

edited

Loading

jbduncan commented Sep 21, 2016 •

edited

Loading

jbduncan commented Sep 24, 2016 •

edited

Loading

jbduncan commented Sep 24, 2016 •

edited

Loading