EmptyByteBuf referencing Cleaner instances that change

[galderz]

Trying Andrew's debug info stuff I encountered this:

Fatal error:com.oracle.svm.core.util.VMError$HostedError: com.oracle.svm.core.util.UserError$UserException: Static field or an object referenced from a static field changed during native image generation?
  object:jdk.internal.ref.Cleaner@3f332391  of class: jdk.internal.ref.Cleaner
  reachable through:
    object: jdk.internal.ref.Cleaner@ab523b6  of class: jdk.internal.ref.Cleaner
    object: jdk.internal.ref.Cleaner@6fe2d54d  of class: jdk.internal.ref.Cleaner
    object: java.nio.DirectByteBuffer[pos=0 lim=0 cap=0]  of class: java.nio.DirectByteBuffer
    root: io.netty.buffer.EmptyByteBuf.nioBuffers(int, int)

	at com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:72)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:647)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:451)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

Adding debug info might be triggering extra GC cycles that results in Cleaner references changing while the heap is being written.

There might be multiple ways of fixing this, but one that have seemed to work for me is to mark EmptyByteBuf.EMPTY_BYTE_BUFFER as a recomputed field with kind Reset.

Other potential solutions (not tried) might be to make EmptyByteBuf a runtime initialized class?

I'll send a PR in a bit.

[galderz]

Longer output

[galderz]

PR with fix.

[galderz]

The solution proposed of recomputed Reset field won't work since that would also null the value of EmptyByteBuf.EMPTY_BYTE_BUFFER at runtime. FromAlias won't work either because that would lead to the value set in the alias to be stored in the heap.

So, I'm now investigating if EmptyByteBuf could be a runtime initialised class.

Another option would to see if the EmptyByteBuf is really needed/used by Quarkus and see if the path to method that makes this class to be stored in the heap can be cut off.

[galderz]

Yet another option would be to patch the Netty code to initialize EMPTY_BYTE_BUFFER lazily, but that's not really desirable. Lazy initialization worked in this particular case.

More options would include exploring whether the Cleaner in DirectByteBuffer can be stopped from changing somehow (e.g. mark it as recompute reset?).

[galderz]

With the current state of things, it'd appear that runtime initialization does not work (see here). No clues after adding -H:+TraceClassInitialization (see here).

I've hooked a debugger to find out why the EMPTY_BYTE_BUFFER gets used during the build time and stacktrace is this:

<clinit>:50, EmptyByteBuf (io.netty.buffer)
<init>:101, AbstractByteBufAllocator (io.netty.buffer)
<init>:74, UnpooledByteBufAllocator (io.netty.buffer)
<init>:59, UnpooledByteBufAllocator (io.netty.buffer)
<init>:46, UnpooledByteBufAllocator (io.netty.buffer)
<clinit>:37, UnpooledByteBufAllocator (io.netty.buffer)
<clinit>:74, Unpooled (io.netty.buffer)
ensureClassInitialized0:-1, Unsafe (jdk.internal.misc)
ensureClassInitialized:1042, Unsafe (jdk.internal.misc)
ensureClassInitialized:698, Unsafe (sun.misc)
ensureClassInitialized:169, ConfigurableClassInitialization (com.oracle.svm.hosted.classinitialization)
computeInitKindAndMaybeInitializeClass:586, ConfigurableClassInitialization (com.oracle.svm.hosted.classinitialization)
computeInitKindAndMaybeInitializeClass:132, ConfigurableClassInitialization (com.oracle.svm.hosted.classinitialization)
maybeInitializeHosted:160, ConfigurableClassInitialization (com.oracle.svm.hosted.classinitialization)
registerType:223, SVMHost (com.oracle.svm.hosted)
createType:264, AnalysisUniverse (com.oracle.graal.pointsto.meta)
lookupAllowUnresolved:205, AnalysisUniverse (com.oracle.graal.pointsto.meta)
lookup:182, AnalysisUniverse (com.oracle.graal.pointsto.meta)
<init>:108, AnalysisMethod (com.oracle.graal.pointsto.meta)
createMethod:412, AnalysisUniverse (com.oracle.graal.pointsto.meta)
lookupAllowUnresolved:400, AnalysisUniverse (com.oracle.graal.pointsto.meta)
lookupMethod:116, WrappedConstantPool (com.oracle.graal.pointsto.infrastructure)
lookupMethodInPool:4323, BytecodeParser (org.graalvm.compiler.java)
lookupMethodInPool:107, SharedGraphBuilderPhase$SharedBytecodeParser (com.oracle.svm.hosted.phases)
lookupMethod:4317, BytecodeParser (org.graalvm.compiler.java)
genInvokeStatic:1659, BytecodeParser (org.graalvm.compiler.java)
processBytecode:5340, BytecodeParser (org.graalvm.compiler.java)
iterateBytecodesForBlock:3423, BytecodeParser (org.graalvm.compiler.java)
processBlock:3230, BytecodeParser (org.graalvm.compiler.java)
build:1088, BytecodeParser (org.graalvm.compiler.java)
buildRootMethod:982, BytecodeParser (org.graalvm.compiler.java)
run:84, GraphBuilderPhase$Instance (org.graalvm.compiler.java)
run:49, Phase (org.graalvm.compiler.phases)
apply:214, BasePhase (org.graalvm.compiler.phases)
apply:42, Phase (org.graalvm.compiler.phases)
apply:38, Phase (org.graalvm.compiler.phases)
parse:225, MethodTypeFlowBuilder (com.oracle.graal.pointsto.flow)
apply:352, MethodTypeFlowBuilder (com.oracle.graal.pointsto.flow)
doParse:322, MethodTypeFlow (com.oracle.graal.pointsto.flow)
ensureParsed:311, MethodTypeFlow (com.oracle.graal.pointsto.flow)
addContext:112, MethodTypeFlow (com.oracle.graal.pointsto.flow)
onObservedUpdate:373, DefaultAnalysisPolicy$DefaultSpecialInvokeTypeFlow (com.oracle.graal.pointsto)
notifyObservers:470, TypeFlow (com.oracle.graal.pointsto.flow)
update:542, TypeFlow (com.oracle.graal.pointsto.flow)
update:151, SourceTypeFlowBase (com.oracle.graal.pointsto.flow)
run:530, BigBang$2 (com.oracle.graal.pointsto)
lambda$execute$0:173, CompletionExecutor (com.oracle.graal.pointsto.util)
run:-1, 112080609 (com.oracle.graal.pointsto.util.CompletionExecutor$$Lambda$362)
exec:1426, ForkJoinTask$RunnableExecuteAction (java.util.concurrent)
doExec:290, ForkJoinTask (java.util.concurrent)
topLevelExec:1020, ForkJoinPool$WorkQueue (java.util.concurrent)
scan:1656, ForkJoinPool (java.util.concurrent)
runWorker:1594, ForkJoinPool (java.util.concurrent)
run:177, ForkJoinWorkerThread (java.util.concurrent)

ByteBufAllocator links to it which brings it to build time. It's also interesting to see how the empty byte memory address is calculated at build time. Seems to me that should be disabled (recompute field to 0), or done at runtime (see here).

The original idea of reset EMPTY_BYTE_BUFFER to null might still work, but it also needs more help to avoid NPE: the memory address should be set to 0, and then substitute any methods that use the empty byte buffer for a new one.

I've also try to use alias to avoid reusing the one that has been used when calculating the memory address but that didn't work.

[galderz]

New PR with a better fix.