Building for Android target not working

[cookiengineer]

I'm not sure how to start, the last couple days I've been trying to build node.js as an Android target, but I've gotten not very far with it.

I was first trying out the android-configure script in this repository which seems heavily outdated, as compiling node --without-snapshot isn't possible since a while now.

Afterwards I was trying to figure out how Termux does it, while trying to build node as static as possible, because their packaging system uses all of the libraries in a shared manner.

Side note: The nodejs-mobile hardfork is totally outdated and uses the abandoned chakra fork, so I guess that cannot be relied on anyhow. Tried that out, nothing works, segfault pretty much everywhere. Still they had the same build issues, and their reasons for the hardfork with all kinds of weird gyp file and #ifdef replacements in the codebase, though it's impossible to say whether these patches still are necessary for upstream node or not.

---

Currently, I've gotten this far, but I still get a segfault on every Android API level. I've got no clue what the issue is (hence the reason for this Bug Report). I cannot build a working node binary and/or libnode.so for Android that works correctly without a segfault.

The current build script looks like this (based on the android-configure script, but modified a bit to be failsafe, because some stuff was missing)... currently I am building for AOSP / OmniROM and Android API Level 28 (aka Android 9.0+).

#!/bin/bash



# XXX: API Levels
# Android 10   -> 29
# Android  9   -> 28
# Android  8.1 -> 27
# Android  8.0 -> 26



export ANDROID_HOME="/opt/android-sdk";
export ANDROID_SDK_VERSION="28";
export ANDROID_NDK="/opt/android-ndk";
export ANDROID_NDK_HOME="/opt/android-ndk";

HOST_OS="linux";
HOST_ARCH="x86_64";
HOST_ROOT="$PWD";



build_node() {

	arch="$1";

	if [[ "$arch" == "armeabi-v7a" ]]; then
		DEST_CPU="arm";
		TOOLCHAIN_NAME="armv7a-linux-androideabi";
		PLATFORM_NAME="arm-linux-androideabi";
		ARCH="arm";
		# TODO: Verify that ARCH is arm for v8
	elif [[ "$arch" == "arm64-v8a" ]]; then
		DEST_CPU="arm64";
		TOOLCHAIN_NAME="aarch64-linux-android";
		PLATFORM_NAME="aarch64-linux-androideabi";
		ARCH="arm64";
	elif [[ "$arch" == "x86" ]]; then
		DEST_CPU="ia32";
		TOOLCHAIN_NAME="i686-linux-android";
		PLATFORM_NAME="i686-linux-android";
		ARCH="ia32";
	elif [[ "$arch" == "x86_64" ]]; then
		DEST_CPU="x64";
		TOOLCHAIN_NAME="x86_64-linux-android";
		PLATFORM_NAME="x86_64-linux-android";
		ARCH="x64";
	fi;

	if [[ "$DEST_CPU" != "" ]]; then

		export GYP_DEFINES="target_arch=$ARCH v8_target_arch=$ARCH android_target_arch=$ARCH host_os=$HOST_OS OS=android";

		export CC_host="$(which clang)";
		export CXX_host="$(which clang++)";
		export AR_host="$(which ar)";
		export AS_host="$(which as)";
		export LD_host="$(which ld)";
		export NM_host="$(which nm)";
		export RANLIB_host="$(which ranlib)";
		export STRIP_host="$(which strip)";

		TOOLCHAIN_PATH="$ANDROID_NDK/toolchains/llvm/prebuilt/$HOST_OS-$HOST_ARCH";
		export PATH="$TOOLCHAIN_PATH/bin:$PATH";

		export CC_target="$TOOLCHAIN_PATH/bin/$TOOLCHAIN_NAME$ANDROID_SDK_VERSION-clang";
		export CXX_target="$TOOLCHAIN_PATH/bin/$TOOLCHAIN_NAME$ANDROID_SDK_VERSION-clang++";
		export AR_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-ar";
		export AS_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-as";
		export LD_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-ld";
		export NM_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-nm";
		export RANLIB_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-ranlib";
		export STRIP_target="$TOOLCHAIN_PATH/bin/$PLATFORM_NAME-strip";


		cd "$HOST_ROOT/node";

		chmod +x ./configure;

		./configure \
			--dest-cpu="$DEST_CPU"\
			--dest-os="android" \
			--without-intl \
			--without-inspector \
			--without-node-snapshot \
			--without-node-code-cache \
			--without-npm \
			--without-snapshot \
			--openssl-no-asm \
			--cross-compiling;

		if [[ "$?" == "0" ]]; then

			cd "$HOST_ROOT/node";

			export LDFLAGS=-shared;
			make;


			if [[ -f "$HOST_ROOT/node/out/Release/lib.target/libnode.so" ]]; then

				mkdir -p "$HOST_ROOT/libnode/$arch";
				cp "$HOST_ROOT/node/out/Release/lib.target/libnode.so" "$HOST_ROOT/libnode/$arch/libnode.so";

			fi;

		fi;

		if [[ "$?" == "0" ]]; then

			cd "$HOST_ROOT/node";

			export LDFLAGS=-shared;
			make;


			if [[ -f "$HOST_ROOT/node/out/Release/node" ]]; then

				mkdir -p "$HOST_ROOT/libnode/$arch";
				cp "$HOST_ROOT/node/out/Release/node" "$HOST_ROOT/libnode/$arch/node";

			fi;

		fi;

	fi;

}

# build_node "armeabi-v7a";
build_node "arm64-v8a";
# build_node "x86";
# build_node "x86_64";

My questions are now as following:

• How to correctly build node.js using the correct Android SDK API Level and Android NDK r21?
• How to correctly build node.js for Android as a libnode.so so that it can be used inside Android Apps, too (using the Android NDK)?
• Am I still assuming correctly that libicu problems arise and therefore Intl support has to be disabled? If not, how to fix all the build errors?
• I added all flags during try/error of the compilation process manually, each time I got further in the compilation process another error happened down the line, then I removed that feature accordingly. Meanwhile I probably compiled node's codebase for at least 30 times (since Saturday non-paused) without any working result.
• I've also seen that --parallel is disabled, so basically everything is compiled with -j 1 - why is that?

Thanks in advice.

[cookiengineer]

After looking at the sijtech/build-nodejs-for-android repo I found some ideas what might be causing the issue. I think down the toolchain some host/target exports were missing, especially for ld and ar.

[cookiengineer]

Update: I've compiled now pretty much all constellations of flags I can think of. Every single variant segfaults on an AOSP Android 9 and another device with OmniROM head / Android 10.

I think nothing's actually compiling correctly, what am I missing?

[wsxiaoys]

https://github.com/PixelShiftAI/node-shared-build/actions/runs/169326880

FYI we've built node 14.2.0/14.5.0 successful into shared libraries on Android arm64.

There was some attempts on ia32/arm/x86_64, but not much progress so far.

[cookiengineer]

@wsxiaoys Could you compile only against API level 23 and node v14.2.0? Did you have to change any of the $CONFIGURE_OPTIONS via environment variable?

[wsxiaoys]

This is the 14.2 build on API 23:
https://github.com/PixelShiftAI/node-shared-build/runs/871353578?check_suite_focus=true

CONFIGURATION_OPTIONS is set to --without-npm --with-intl=small-icu --shared

[cookiengineer]

@wsxiaoys: When using your script, I got all kinds of linker problems, because LD_host and LD_target were not set. Which linker are you using? ld? I could not see any linker-relevant variables being set in your build script in the node-shared-build repo.

When I'm setting LD_host to which ld and LD_target to $TOOLCHAIN_PATH/bin/$TOOLCHAIN_NAME-ld it won't work and I'm still getting undefined reference errors.

(for reference)

  /opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++ -rdynamic -fPIC -Wl,-rpath=\$ORIGIN/lib.target/ -Wl,-rpath-link=\/home/cookiengineer/Software/nodejs/out/Release/lib.target/  -o /home/cookiengineer/Software/nodejs/out/Release/embedtest -Wl,--start-group /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/src/node_snapshot_stub.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/src/node_code_cache_stub.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/test/embedding/embedtest.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/libnode.so /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/histogram/libhistogram.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/uvwasi/libuvwasi.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_snapshot.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libplatform.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicui18n.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/zlib/libzlib.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/llhttp/libllhttp.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/cares/libcares.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/uv/libuv.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/nghttp2/libnghttp2.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/brotli/libbrotli.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/openssl/libopenssl.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicuucx.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicudata.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicustubdata.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_base_without_compiler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libbase.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libsampler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_zlib.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_compiler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_initializers.a -Wl,--end-group -lm -ldl -llog
/home/cookiengineer/Software/nodejs/out/Release/obj.target/libnode.so: undefined reference to `uv__random_getentropy'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [embedtest.target.mk:184: /home/cookiengineer/Software/nodejs/out/Release/embedtest] Error 1
make[1]: *** Waiting for unfinished jobs....

[wsxiaoys]

@wsxiaoys: When using your script, I got all kinds of linker problems, because LD_host and LD_target were not set. Which linker are you using? ld? I could not see any linker-relevant variables being set in your build script in the node-shared-build repo.

When I'm setting LD_host to which ld and LD_target to $TOOLCHAIN_PATH/bin/$TOOLCHAIN_NAME-ld it won't work and I'm still getting undefined reference errors.

(for reference)

  /opt/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android28-clang++ -rdynamic -fPIC -Wl,-rpath=\$ORIGIN/lib.target/ -Wl,-rpath-link=\/home/cookiengineer/Software/nodejs/out/Release/lib.target/  -o /home/cookiengineer/Software/nodejs/out/Release/embedtest -Wl,--start-group /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/src/node_snapshot_stub.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/src/node_code_cache_stub.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/embedtest/test/embedding/embedtest.o /home/cookiengineer/Software/nodejs/out/Release/obj.target/libnode.so /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/histogram/libhistogram.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/uvwasi/libuvwasi.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_snapshot.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libplatform.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicui18n.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/zlib/libzlib.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/llhttp/libllhttp.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/cares/libcares.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/uv/libuv.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/nghttp2/libnghttp2.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/brotli/libbrotli.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/deps/openssl/libopenssl.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicuucx.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicudata.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/icu/libicustubdata.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_base_without_compiler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libbase.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_libsampler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_zlib.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_compiler.a /home/cookiengineer/Software/nodejs/out/Release/obj.target/tools/v8_gypfiles/libv8_initializers.a -Wl,--end-group -lm -ldl -llog
/home/cookiengineer/Software/nodejs/out/Release/obj.target/libnode.so: undefined reference to `uv__random_getentropy'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [embedtest.target.mk:184: /home/cookiengineer/Software/nodejs/out/Release/embedtest] Error 1
make[1]: *** Waiting for unfinished jobs....

Yes it just use the default ld.

Guessing you could just pull the official github action image and try the script inside and compare the environment difference?

[cookiengineer]

@wsxiaoys The issue is that a build with only "LD=which ld" will lead to a successful build, but a segfaulting node binary.

Did you verify on Android that the binary actually works without a segfault?

It seems, when using ldd ./node in Termux, that the binary has the following linked libraries:

• libnode.so (included in same directory and available via ".../node/out/Release/lib.target/libnode.so")
• libm.so (available via /system/lib/libm.so)
• libdl.so (which probably should be /system/lib/libdl_android.so)
• libc++_shared.so (NOT available, only available via Android NDK, see here)
• libc.so (available via /system/lib/libc.so)

I tried copying the libc++_shared.so from /opt/android-ndk/sources/cxx-stl/llvm-libc++/libs/arm64-v8a/libc++_shared.so to the device, but that didn't resolve the segfault either.

The resulting binary doesn't even try to read any file, it just segfaults immediately. When using gdb to debug what's going on, there's not a single file i/o.

[wsxiaoys]

@cookiengineer Hi, our use case is to embed node as a shared library (thus only libnode.so is used). So I'm not really familiar with the shell use case presented in your screenshot.

[stormagain]

@cookiengineer It's maybe the ANDROID_SDK_VERSION=28 not support on your device. is your device android sdk version >= 28 ?

[CMCDragonkai]

The nodejs-mobile is now using V8 jitless mode according to https://www.janeasystems.com/blog/significant-updates-in-the-new-release-of-node-js-mobile/.

[cookiengineer]

@stormagain The device I used for testing was Android 10 at the time (Android 9 is SDK level 28 iirc), so it should've worked.

Gonna try to take a look at the nodejs-mobile repo to figure out what they do differently than upstream. Last time I took a look at it, they were using a heavily outdated nodejs LTS variant which was not reproducible in terms of compiling it yourself.

[xeoshow]

Just would like to know: Is the android-configure from the root path of node project (https://github.com/nodejs/node/blob/main/android-configure) just for building nodejs for android? Seems no any document telling this? Thanks a lot.