ADR for managing native libraries in GraalVM native images

This commit introduces an ADR detailing the process for managing native libraries in Quarkus extensions when performing native compilation with GraalVM.

Key points include:
- Using GraalVM's `Feature` mechanism to select and include platform-specific native libraries.
- Centralizing native library management and runtime initialization configuration within the `Feature`.
- Examples for implementing a `Feature` to include native libraries such as `brotli.so`.
- Discussion of considered options and consequences of using the `Feature` approach.

Co-authored-by: @zakkak

adr/0006-native-compilation-with-binary-libraries.adoc

@@ -0,0 +1,155 @@
+= Including Native Libraries in the Native Image
+
+* Status: _Proposed_
+* Date: 2024-10-21
+* Authors: @cescoffier, @zakkak
+
+== Context and Problem Statement
+
+At Quarkus’ scale, several extensions integrate libraries that require native code.
+Most of these libraries rely on JNI (Java Native Interface) to interact with native code.
+Consequently, these native libraries must be included in the native image, and in some cases, the loading mechanism must be adapted for a native compilation context.
+
+For example, the Kafka Streams extension requires `librocksdbjni.so`, and Vert.x HTTP needs `brotli.so` to be included in the native image (to handle compression).
+
+In JVM mode, these libraries are present on the classpath, and the JVM automatically manages their loading.
+However, in native mode, we must ensure that the correct version of the native library — matching the target platform (OS/architecture) — is bundled into the native image.
+
+This ADR aims to define a standardized approach to managing and including native libraries in the native image for Quarkus extensions.
+
+== Including Native Libraries Using GraalVM Feature
+
+One of the main requirements is selecting the native library version that matches the target platform.
+For example, when compiling for `Linux/x86_64`, we must include the `Linux/x86_64` version of the library in the native image.
+
+This selection cannot be efficiently handled in Quarkus’ build steps because platform ambiguity can arise (more on this in the next section). Therefore, we propose using GraalVM’s `Feature` mechanism to manage native libraries.
+
+A `Feature` allows extending the native image generation process.
+It can accurately determine the host platform (since GraalVM doesn’t support cross-compilation) and include the appropriate native library in the image.
+
+Additionally, `Feature` allows configuring various aspects of the native compilation process.
+This enables encapsulating all logic related to native library management in a single location.
+
+The `Feature` implementation is located in the _runtime_ module of the extension, typically in a dedicated `graal` package (which contains the `Feature` and any necessary substitutions).
+
+Here’s an example of a Feature implementation:
+
+[source,java]
+----
+package io.quarkus.myextension.runtime.graal; // <1>
+
+import org.graalvm.nativeimage.hosted.Feature; // <2>
+import org.graalvm.nativeimage.hosted.RuntimeClassInitialization;
+import org.graalvm.nativeimage.hosted.RuntimeResourceAccess;
+
+public class MyFeature implements Feature {
+
+    @Override
+    public void beforeAnalysis(BeforeAnalysisAccess access) { // <3>
+        // Decide which native library to include based on the target platform
+        // `access` allows adding resources to the native image and configuring
+        // classes for runtime initialization.
+        // ...
+    }
+
+}
+----
+1.	The package containing the Feature implementation.
+2.	Importing the necessary classes from the GraalVM SDK.
+3.	The primary method to implement is beforeAnalysis, which runs before the analysis phase of the native image generation process.
+
+The `Feature` interface is part of the GraalVM SDK and exposes several additional methods.
+To use it, we need to add the following dependency:
+
+[source,xml]
+----
+<dependency>
+    <groupId>org.graalvm.sdk</groupId>
+    <artifactId>nativeimage</artifactId>
+    <scope>provided</scope>
+</dependency>
+----
+
+During the `beforeAnalysis` phase, we can add native libraries and resources to the native executable.
+This is also the point where classes needing runtime initialization can be configured.
+
+Here’s an example that includes `brotli.so` in the native image:
+
+[source,java]
+----
+@Override
+public void beforeAnalysis(BeforeAnalysisAccess access) {
+    String nativeLibName = System.mapLibraryName("brotli");
+    String libPath = "lib/" + getPlatform() + "/" + nativeLibName; // <1>
+    RuntimeResourceAccess.addResource(Brotli4jFeature.class.getModule(), libPath); // <2>
+
+    RuntimeResourceAccess.addResource(Brotli4jFeature.class.getModule(),
+    "META-INF/services/com.aayushatharva.brotli4j.service.BrotliNativeProvider"); // <3>
+
+    RuntimeClassInitialization.initializeAtRunTime("com.aayushatharva.brotli4j.Brotli4jLoader"); // <4>
+
+    // ...
+}
+----
+1.	`getPlatform()` determines the host platform (e.g., Linux/x86_64).
+2.	Adds the appropriate native library to the native image.
+3.	Adds additional required resources (here, an SPI).
+4.	Configures the `Brotli4jLoader` class for runtime initialization to avoid issues with static initialization at build time.
+
+Once the `Feature` for a native library is implemented, it must be enabled during the Quarkus build process.
+In the extension processor, a build step should produce a NativeImageFeatureBuildItem to enable the feature:
+
+[source,java]
+----
+// Simplified code
+@BuildStep
+NativeImageFeatureBuildItem enableBrotliFeature() {
+    return new NativeImageFeatureBuildItem(Brotli4jFeature.class.getName()); // <1>
+}
+----
+1.	This step enables the feature in the native image generation process.
+
+Several examples of extensions using this approach are available in the Quarkus codebase:
+
+- https://github.com/quarkusio/quarkus/pull/43828[Brotli]
+- https://github.com/quarkusio/quarkus/pull/43905[Snappy]
+- https://github.com/quarkusio/quarkus/pull/43782[RocksDB]
+
+== Considered Options
+
+=== Option 1: Using Build Steps in the Extension Processor
+
+Initially, native library management was handled through build steps in the extension processor.
+However, this approach couldn’t effectively select the correct native library for the target platform, leading to potential mismatches.
+
+Indeed, the native compilation can run:
+
+- directly on the host
+- in a container (used to produce native images that can be deployed in Linux container)
+
+In the latter case, the target platform is the container’s platform, not the host’s.
+Using `Feature` allows us to accurately determine the target platform and include the appropriate native library.
+Indeed, the `Feature` is executed as part of the native image generation process, which in the latter case, it the container.
+
+Additionally, this method mixed native library management with other build logic, reducing the clarity of the code.
+
+=== Option 2: Using a Dedicated Processor
+
+This approach moves native library management to a dedicated processor, but it still doesn’t fully resolve platform ambiguity.
+
+== Consequences
+
+=== Positive
+
+Encapsulating native library management in a Feature offers several advantages:
+
+* Centralizes all logic related to native library handling.
+* Removes ambiguity regarding the target platform.
+* Provides flexibility in configuring native compilation.
+* Clearly separates concerns, encapsulating native library management.
+
+=== Negative
+
+* Separates native library management from the main extension processor code, potentially requiring developers to look in multiple places for extension logic.
+Especially because the `Feature` is a _build time_ concern, located in the _runtime_ module.
+* Requires additional dependencies to use the GraalVM SDK.