-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[native-image] Classes with loops inside Kotlin coroutines fail to generate native code #366
Comments
Hi @rhencke, thank you for the report and minimal reproducer. At the moment, Graal does not support compilation of non-reducible loops (i.e., loops with multiple entry-points). This is fine on HotSpot as we can just delegate execution of such methods to either the interpreter or C1 and they are fairly rare (for example javac or scalac would never produce such code patterns). Structured loops at an important (an conscious) decision in the design of the Graal IR so i don't think we want to change that. However there are ways we could improve support for non-reducible loops that appear in bytecodes. In particular we could select the entry point that we want to keep and then duplicate parts of the loop on the other entry path so that a single entry point remains. This would cause some increase in code size but at least it would allow Graal to better support Kotlin coroutines . |
I was trying to convert my ktor.io app to a native-image, but ktor uses kotlin coroutines under the hood. Stuck like the issue reporter unfortunately. |
Also encountered with Ktor Kotlin project, Graal RC10. |
I also encountered this with Ktor, and ended up creating a standalone project to do as @gilles-duboscq suggested and duplicate the irreducible loops. It should work in general for any Kotlin coroutines, not just Ktor applications, though a minimal Ktor application is included as an example. I've released it under MIT license: https://github.com/HewlettPackard/kraal It would still be nice if it GraalVM or kotlinc could resolve this independently so they work together out of the box, but maybe others will find this useful as a stop-gap until then. |
I encounterd the same issue. Is this something graal native-image plans to fix in future? |
Use Kraal to prevent GraalVM native-image from failing with "Non-reducible loop". oracle/graal#366 https://github.com/HewlettPackard/kraal Without Kraal, here is the native-image's stacktrace: Error: Non-reducible loop Call path from entry point to pullpitok.AppKt$main$1.invokeSuspend(Object): at pullpitok.AppKt$main$1.invokeSuspend(App.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32) at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285) at java.lang.Shutdown.runHooks(Shutdown.java:123) at java.lang.Shutdown.sequence(Shutdown.java:167) at java.lang.Shutdown.shutdown(Shutdown.java:234) at com.oracle.svm.core.jdk.RuntimeSupport.shutdown(RuntimeSupport.java:181) at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:177) at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0) Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Non-reducible loop at org.graalvm.compiler.java.BciBlockMapping.computeBlockOrder(BciBlockMapping.java:882) at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:527) at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109) at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:806) at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:784) at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95) at org.graalvm.compiler.phases.Phase.run(Phase.java:49) at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197) at org.graalvm.compiler.phases.Phase.apply(Phase.java:42) at org.graalvm.compiler.phases.Phase.apply(Phase.java:38) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:205) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:324) at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310) at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300) at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107) at com.oracle.graal.pointsto.flow.SpecialInvokeTypeFlow.onObservedUpdate(InvokeTypeFlow.java:421) at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:347) at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:389) at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508) at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174) at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Ktor brings a sophisticated asynchronous HTTP client that uses Kotlin coroutines, but all we need is a simple HTTP client for GET requests. Removing coroutines also makes pullpitoK compatible with GraalVM that may be used later for (see #4 and oracle/graal#366).
Use Kraal to prevent GraalVM native-image from failing with "Non-reducible loop". oracle/graal#366 https://github.com/HewlettPackard/kraal Without Kraal, here is the native-image's stacktrace: Error: Non-reducible loop Call path from entry point to pullpitok.AppKt$main$1.invokeSuspend(Object): at pullpitok.AppKt$main$1.invokeSuspend(App.kt) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32) at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:285) at java.lang.Shutdown.runHooks(Shutdown.java:123) at java.lang.Shutdown.sequence(Shutdown.java:167) at java.lang.Shutdown.shutdown(Shutdown.java:234) at com.oracle.svm.core.jdk.RuntimeSupport.shutdown(RuntimeSupport.java:181) at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:177) at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0) Original exception that caused the problem: org.graalvm.compiler.core.common.PermanentBailoutException: Non-reducible loop at org.graalvm.compiler.java.BciBlockMapping.computeBlockOrder(BciBlockMapping.java:882) at org.graalvm.compiler.java.BciBlockMapping.build(BciBlockMapping.java:527) at org.graalvm.compiler.java.BciBlockMapping.create(BciBlockMapping.java:1109) at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:806) at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:784) at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95) at org.graalvm.compiler.phases.Phase.run(Phase.java:49) at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197) at org.graalvm.compiler.phases.Phase.apply(Phase.java:42) at org.graalvm.compiler.phases.Phase.apply(Phase.java:38) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:205) at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:324) at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310) at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300) at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107) at com.oracle.graal.pointsto.flow.SpecialInvokeTypeFlow.onObservedUpdate(InvokeTypeFlow.java:421) at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:347) at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:389) at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508) at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174) at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Ktor brings a sophisticated asynchronous HTTP client that uses Kotlin coroutines, but all we need is a simple HTTP client for GET requests. Removing coroutines also makes pullpitoK compatible with GraalVM that may be used later (see #4 and oracle/graal#366).
Ktor brings a sophisticated asynchronous HTTP client that uses Kotlin coroutines, but all we need is a simple HTTP client for GET requests. Removing coroutines also makes pullpitoK compatible with GraalVM that may be used later (see #4 and oracle/graal#366).
I'm implementing a potential fix in #1330, if you have cases you'd like me to test, please share them in either issue. |
Having it with just kotlin sequences in kotlin 1.3.31:
Does it mean kotlin sequenece is a blocker for graalvm? |
The first CLI parameter is now interpreted as a comma-separated list of repositories. Not implemented with a Kotlin coroutine channel since GraalVM is not compatible with coroutines yet. See oracle/graal#366.
The first CLI parameter is now interpreted as a comma-separated list of repositories. Not implemented with a Kotlin coroutine channel since GraalVM is not compatible with coroutines yet. See oracle/graal#366.
The first CLI parameter is now interpreted as a comma-separated list of repositories. Not implemented with a Kotlin coroutine channel since GraalVM is not compatible with coroutines yet. See oracle/graal#366.
I tried to compile a Spring Boot 2.2 application with WebFlux and Coroutines, but it failed with a similar exception:
|
@gilles-duboscq any further updates on this subject? It is really limiting for kotlin / graal native apps to the point that our team is considering giving up on the stack. |
I'll try to get this into 20.1 otherwise i'll just push transfer that to someone else. |
Great news. Please let us know if there is anything that can be done to help from the community. |
@gilles-duboscq, having this in 20.1 would be great! Keeping my fingers crossed... 🤞🏼 |
This should be fixed by 4662877. The fix is included in the latest 20.1 dev build (e.g., 20.1.0-dev-20200325_0537). I tested that the original example from @rhencke now produces a working native-image. |
* Bump default graal version to 20.2.0 This fixes a problem with projects that make use of the kotlin coroutines. More details: oracle/graal#366 * Add changelog
Consider the following Kotlin code:
Compile it with the Kotlin compiler (kotlin-compiler-1.2.40.zip). (Assume the zip is extracted to a directory represented by
$KOTLIN_HOME
)This will fail to generate a native image, showing the following error:
If you adjust the sample, and comment out the loop within the coroutine as follows, the code successfully generates a working native image:
The text was updated successfully, but these errors were encountered: