Improve GraalVM --shared support for embedding Quarkus native into C/C++ applications

[murphye]

Description

This is an exciting new use case for using Quarkus. By using GraalVM native-image --shared, instead of producing an executable, it can produce a shared library (.so) and header files to compile and link to a C or C++ application. See this documentation: https://www.graalvm.org/reference-manual/native-image/#build-a-shared-library

I have a mostly working example of using native-image --shared with Quarkus to natively build the Saxon Java library, and then use it in a C++ application:
https://github.com/murphye/saxon-graalvm-native-from-cpp/tree/quarkus-issue

Here are some relevant files that you can examine on how this works:

• https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/full-build.sh
• https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/saxon-lib/src/main/java/io/solo/Main.java
• https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/transform-app/transform.cc

Here is the flow for how this works:

Java code -> Quarkus Maven Build -> Quarkus Augmentation -> GraalVM Native Image -> Shared Object and Headers -> C/C++ code -> C/C++ Compiler/Linker -> Executable

With my example, I have this working with a few caveats:

1. Quarkus Maven Plugin Expects an executable (runner) and not an .so. So you get this error:

[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.11.1.Final:build (default) on project saxon-graalvm-native-from-cpp: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]         [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to build native image
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:307)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] Caused by: java.nio.file.NoSuchFileException: /home/eric/GitHub/saxon-graalvm-native-from-cpp/saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native
[ERROR]         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
[ERROR]         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
[ERROR]         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
[ERROR]         at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
[ERROR]         at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
[ERROR]         at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
[ERROR]         at java.base/java.nio.file.Files.readAttributes(Files.java:1764)
[ERROR]         at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
[ERROR]         at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
[ERROR]         at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
[ERROR]         at java.base/java.nio.file.Files.walkFileTree(Files.java:2717)
[ERROR]         at io.quarkus.bootstrap.util.IoUtils.copy(IoUtils.java:126)
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:281)
[ERROR]         ... 10 more

2. Have to manually extract the artifacts from the native-image-source-jar

rm ./saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native.jar
cp ./saxon-lib/target/libsaxon-native-image-source-jar/graal_isolate* transform-app
cp ./saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native* transform-app

3. Need to manually set config for

<quarkus.native.additional-build-args>--shared

also:

quarkus.package.runner-suffix=-native
quarkus.native.enable-http-url-handler=false

4. Have to include @QuarkusMain when it's not really a main entry point:

@QuarkusMain
class Main {

    public static void main(String... args) {}

About my Use Case

I am working on adding extensions to the Envoy proxy, which is written in C++. There are use cases, such as parsing and transforming XML where a Java library is much more suited to anything that is available in C/C++. So, having the ability to build Java to a .so for use with C++ is a boon. Also Quarkus can make this process much easier, and potentially even leverage Quarkus extensions (or build new ones) to support native library use cases. For example, there could be a Quarkus Extension for Saxon/Java that is primarily targeting use as a native library (but could be dual purpose?)

Implementation ideas

These points align to the points above.

1. Update Quarkus Maven Plugin change to not fail when .so and .h output from native-image --shared. This is the most annoying problem right now. If you have any ideas on a workaround, please let me know!

2. Quarkus Maven Plugin change to copy .so and .h output from native-image --shared into a directory like target/libsaxon in this case. Make path configurable to potentially copy to ../../where_my_c_code_is

3. Create some new default Quarkus configuration for building shared libraries. Have some configuration flag based on Add doc on how to run GraalTest based tests from IDE #4?

4. Create a new annotation such as @NativeLibrary rather than use @QuarkusMain as a workaround.

5. Enforce that the compiled library name starts with lib so it can be linked properly from C/C++.

[gsmet]

I think what you need is probably an entirely new quarkus.package.type.

…

On Thu, Feb 11, 2021 at 8:12 PM Eric Murphy ***@***.***> wrote: Description This is an exciting new use case for using Quarkus. By using GraalVM native-image --shared, instead of producing an executable, it can produce a shared library (.so) and header files to compile and link to a C or C++ application. See this documentation: https://www.graalvm.org/reference-manual/native-image/#build-a-shared-library I have a mostly working example of using native-image --shared with Quarkus to natively build the Saxon Java library, and then use it in a C++ application: https://github.com/murphye/saxon-graalvm-native-from-cpp/tree/quarkus-issue Here are some relevant files that you can examine on how this works: - https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/full-build.sh - https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/saxon-lib/src/main/java/io/solo/Main.java - https://github.com/murphye/saxon-graalvm-native-from-cpp/blob/quarkus-issue/transform-app/transform.cc Here is the flow for how this works: Java code -> Quarkus Maven Build -> Quarkus Augmentation -> GraalVM Native Image -> Shared Object and Headers -> C/C++ code -> C/C++ Compiler/Linker -> Executable With my example, I have this working with a few caveats: 1. Quarkus Maven Plugin Expects an executable (runner) and not an .so. So you get this error: [ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.11.1.Final:build (default) on project saxon-graalvm-native-from-cpp: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [ERROR] [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to build native image [ERROR] at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:307) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566) [ERROR] at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920) [ERROR] at io.quarkus.builder.BuildContext.run(BuildContext.java:277) [ERROR] at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415) [ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452) [ERROR] at java.base/java.lang.Thread.run(Thread.java:834) [ERROR] at org.jboss.threads.JBossThread.run(JBossThread.java:501) [ERROR] Caused by: java.nio.file.NoSuchFileException: /home/eric/GitHub/saxon-graalvm-native-from-cpp/saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native [ERROR] at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) [ERROR] at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) [ERROR] at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) [ERROR] at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) [ERROR] at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149) [ERROR] at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) [ERROR] at java.base/java.nio.file.Files.readAttributes(Files.java:1764) [ERROR] at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225) [ERROR] at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) [ERROR] at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) [ERROR] at java.base/java.nio.file.Files.walkFileTree(Files.java:2717) [ERROR] at io.quarkus.bootstrap.util.IoUtils.copy(IoUtils.java:126) [ERROR] at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:281) [ERROR] ... 10 more 1. Have to manually extract the artifacts from the native-image-source-jar rm ./saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native.jar cp ./saxon-lib/target/libsaxon-native-image-source-jar/graal_isolate* transform-app cp ./saxon-lib/target/libsaxon-native-image-source-jar/libsaxon-native* transform-app 1. Need to manually set config for <quarkus.native.additional-build-args>--shared also: quarkus.package.runner-suffix=-native quarkus.native.enable-http-url-handler=false 1. Have to include @QuarkusMain when it's not really a main entry point: @QuarkusMain class Main { public static void main(String... args) {} About my Use Case I am working on adding extensions to the Envoy proxy, which is written in C++. There are use cases, such as parsing and transforming XML where a Java library is much more suited to anything that is available in C/C++. So, having the ability to build Java to a .so for use with C++ is a boon. Also Quarkus can make this process much easier, and potentially even leverage Quarkus extensions (or build new ones) to support native library use cases. For example, there could be a Quarkus Extension for Saxon/Java that is primarily targeting use as a native library (but could be dual purpose?) Implementation ideas These points align to the points above. 1. Update Quarkus Maven Plugin change to not fail when .so and .h output from native-image --shared. This is the most annoying problem right now. If you have any ideas on a workaround, please let me know! 2. Quarkus Maven Plugin change to copy .so and .h output from native-image --shared into a *directory* like target/libsaxon in this case. Make path configurable to potentially copy to ../../where_my_c_code_is 3. Create some new default Quarkus configuration for building shared libraries. Have some configuration flag based on #4 <#4>? 4. Create a new annotation such as @NativeLibrary rather than use @QuarkusMain as a workaround. 5. Enforce that the compiled library name starts with lib so it can be linked properly from C/C++. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#15012>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJYOBJ3K6IM45GDVH4XRLTS6QTZNANCNFSM4XPMJ37A> .

[murphye]

Yes, I would agree with that. It could be something like:

<quarkus.package.type>native-library</quarkus.package.type>

[magick93]

This sounds like a fantastic idea! I hope it gets added as a new quarkus project type.

[mkouba]

@Sanne @galderz @Karm @geoand @zakkak @aloubyansky WDYT?

[aloubyansky]

Great idea

[Sanne]

Certainly interesting from a perspective of "engineers love puzzles" - but could you expand on compelling use cases?

[mkouba]

Certainly interesting from a perspective of "engineers love puzzles" - but could you expand on compelling use cases?

@murphye described one use case in the description ;-)

[Sanne]

ah yes let me clarify my request for more compelling use cases: I get it that embedding a Java library like an XML parser can be a boon to a C developer, but that's about embedding specific libraries only - you might be better off in doing so with just GraalVM.

My question is to expand on the use cases to embed a whole Quarkus stack - which often has a focus on controlling the bootstrap, integrate all components.

In some ways Quarkus has outgrown its identity and we should think about it; it's both of:

• an all-batteries included stack, opinionated but easy to start with for newcomers
• a sophysticated build tool which advanced developers can use to produce highly optimized builds for both JVM and native

These are effectively two different layers; our documentation, project identity and goals are intertwined today but I feel that some advanced users would prefer to use only the low level layer, to build different opinionated stacks on top.

[Karm]

I don't think including general purpose libs like those for XML/JSON manipulation in a C app from Java world via native-image would be a boon. It is slower and more CPU hungry than JVM mode, so it is likely to be also less efficient than almost anything remotely competently written in C++. I am somewhat worried about this Envoy useage @murphye 🤷‍♂️

On the other hand, there are probably projects that we maintain primarily in Java and somewhat (perhaps ? 😃 ) begrudgingly port to C/C++ to interface with the wider audience.

https://github.com/infinispan/cpp-client comes to my mind.

So, what if we could use this to maintain the primary code base in Java for those projects where Java is the main tool anyway and to build shared native libs for clients instead of porting them to C++?

Quarkus would act as a "mere" build toolchain for this.

Disclaimer: I used ISPN cpp-client once years ago and I am not an expert by any stretch of the imagination. It might be a vary bad idea after all...

[Sanne]

I don't think including general purpose libs like those for XML/JSON manipulation in a C app from Java world via native-image would be a boon. It is slower and more CPU hungry than JVM mode, so it is likely to be also less efficient than almost anything remotely competently written in C++.

Sure it's not granted that any Java library would be more efficient than one in C, but the opposite also holds - so there might be situations in which this could actually be an economically wise choice over rewriting it all in C. Of course I agree one need to be cautious and verify assumptions, but it might be a smart choice in some cases - and this is about giving such an option (not to recommend it as a general goal).

https://github.com/infinispan/cpp-client comes to my mind.

That's an excellent example; I'm sure the Infinispan team would prefer to be able to reuse some code.

However, I question about the need for this being a Quarkus feature - over say a suggestion to the Infinispan team to use GraalVM natively compiled .so libraries.

So I'll reiterate my question - I think we need use cases which are specific about Quarkus.

[murphye]

@Sanne The final implementation of my use case with Saxon used GraalVM native-image without Quarkus. It was used in the Solo.io Gloo products.

So, I would say this Quarkus feature is "nice to have" at most and probably requires further consideration as you suggest. Maybe there are other use cases to justify this. I am not sure.

Quarkus would have added value if there was already a Saxon Quarkus extension and I didn't have to write a bunch of GraalVM config to make it work. Maybe this is a rare scenario anyways.

[Sanne]

@murphye understood - yes I agree it would be nice to have it. You had me on the Infinispan cpp-client :)

cc/ @n1hility

[whisper-bye]

+1

[magick93]

Is there any development on this?

Having such a feature could open up some interesting possibilities, for example, automatically generating integrations with node, such as what Rust does with https://napi.rs/