Replace gradle-node-plugin with own tasks

[mbland]

Removing gradle-node-plugin cuts the project's dependency footprint slightly, and enables Windows builds to succeed.

Executing pnpm from a Gradle task proved straightforward. Also added a frontendInstall task, removed the separate "Install Node packages" step from run-tests.yaml, and refined inputs and outputs for more precise dependency tracking.

Specifically learned about using outputs.upToDateWhen { true } as a means to prevent frontendInstall from always running from:

• https://discuss.gradle.org/t/why-does-my-task-run-even-though-non-of-its-inputs-has-changed/5350/2

---

I entertained the notion of fixing Windows builds by removing gradle-node-plugin after a couple of days of trying to fix that plugin.

The moral of the following story:

• Sometimes you really shouldn't add a dependency if you can do what you need to do without it.

When starting the project, I thought running Node.js from Gradle might be tricky, so I searched for a plugin. As it turned out, node-gradle/gradle-node-plugin was available, and seemed to work well on macOS and Linux. However, it broke the project on Windows, because it failed trying to execute uname on Windows.

I got the plugin successfully working on Windows by introducing the following PowerShell command to replace uname (in
com.github.gradle.node.NodePlugin.addPlatform()):

if (name.toLowerCase().contains("windows")) {
    executable = "powershell.exe"
    args = listOf(
        "-Command",
        "& {Get-CimInstance Win32_OperatingSystem | " +
            "Select-Object -ExpandProperty OSArchitecture}"
    )
}

I also updated com.github.gradle.node.util.parseOsArch to handle the "ARM 64-bit Processor" value returned by the command.

However, many tests continued to fail, which I realized was because no win-arm64 build of Node.js was available for the default Node.js version 18.17.1:

• https://nodejs.org/dist/v18.17.1/

So I updated the default to 20.10.0 (and the npm version to 10.2.3, which ships with 20.10.0). All the (very slow) tests began to pass except for NpmProxy_integTest and YarnProxy_integTest.

After some creative debugging whereby I emitted the test-supplied values for HTTP_PROXY and HTTPS_PROXY, I could see that Node couldn't see the test-supplied values. Some digging revealed that past versions of Gradle and its TestKit didn't have native support for arm64 that would allow for setting environment variables:

• TestKit does not pass environment variables for older Gradle versions when running on M1 mac gradle/gradle#22012

It seemed that Gradle 7.5.1, the default used by gradle-node-plugin, should be OK. However, it didn't seem to be.

And this is where the despair really started kicking in.

I tried various combinations of updating the JDK from 11 to 17.0.9, Gradle from 7.5.1 to 8.5, and updating the build.gradle.kts file.

I thought updating to 17.0.9 might solve some problems, and it seemed as high as I could go to still suport 7.5.1:

• https://docs.gradle.org/current/userguide/compatibility.html

One minor change to the build file involved moving merging entries from the deprecated pluginBundle block to the existing gradlePlugin block:

• https://docs.gradle.org/current/userguide/upgrading_version_7.html#pluginbundle_dropped_in_plugin_publish_plugin

Another involved disabling the com.cinnober.gradle.semver-git plugin, because launching subprocesses during the configuration phase is now an error when using the configuration cache:

• https://docs.gradle.org/current/userguide/upgrading_version_7.html#use_providers_to_run_external_processes
• https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:requirements:external_processes

I also deleted the ill advised and useless org.gradle.test-retry plugin and its configuration. The tests didn't appear to be unstable, so the tests would run multiple times to keep failing in the same way. If they didn't fail the same way, or sometimes passed, that'd've been even worse. See:

• https://mike-bland.com/2023/09/13/the-inverted-test-pyramid.html

Then the build would fail while compiling Kotlin:

  API version 1.3 is no longer supported; please, use version 1.4 or
  greater.

So I set in the build file:

tasks.compileKotlin {
    kotlinOptions {
        apiVersion = "1.9"
        languageVersion = "1.9"

I also boosted IntelliJ IDEA "Settings > Build, Execution, Deployment > Compiler > Kotlin Compiler > Target JVM Version" setting to 17. Then I updated the build file variable:

val compatibilityVersion = JavaVersion.VERSION_17

And then, finally, tests still failed because something funny happened inside the Spock test framework, with a bunch of errors like:

SystemVersion_integTest > use system node version and exec node, npm, npx and yarn program (#gv.version) > use system node version and exec node, npm, npx and yarn program (8.5) FAILED
    java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Map java.util.Collections$UnmodifiableMap.m accessible: module java.base does not "opens java.util" to unnamed module @609db546
        at org.junit.contrib.java.lang.system.EnvironmentVariables.getFieldValue(EnvironmentVariables.java:188)
        at org.junit.contrib.java.lang.system.EnvironmentVariables.getEditableMapOfVariables(EnvironmentVariables.java:150)
        at org.junit.contrib.java.lang.system.EnvironmentVariables.access$200(EnvironmentVariables.java:49)
        at org.junit.contrib.java.lang.system.EnvironmentVariables$EnvironmentVariablesStatement.restoreOriginalVariables(EnvironmentVariables.java:134)
        at org.junit.contrib.java.lang.system.EnvironmentVariables$EnvironmentVariablesStatement.evaluate(EnvironmentVariables.java:125)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
        at org.spockframework.junit4.AbstractRuleInterceptor.evaluateStatement(AbstractRuleInterceptor.java:66)
    [ ...snip... ]

I already decided at this point to get rid of gradle-node-plugin, and developed this change. But then I decided to go a little further.

This appears to be related to:

• Cannot create mock due to module java.base does not "opens java.lang.invoke" spockframework/spock#1406

This appears to be a consequence of:

• JEP 396: Strongly Encapsulate JDK Internals by Default https://openjdk.org/jeps/396

This apparently shipped in Java 16, and people noticed in in Spock after upgrading to Java 17.

Based on the information in the spock issue, I tried adding the following jvmArgs entry in the build file:

tasks.withType(Test::class) {
    useJUnitPlatform()
    systemProperty(
        // ...snip...
    )
    jvmArgs("--add-opens", "java.base/java.util=ALL-UNNAMED")

Reference for --add-opens:

• https://docs.oracle.com/en/java/javase/17/migrate/migrating-jdk-8-later-jdk-releases.html#GUID-12F945EB-71D6-46AF-8C3D-D354FD0B1781

This actually fixed the error, but on macOS, three tests failed with:

> Task :npmInstall FAILED
npm WARN tarball tarball data for npm@https://registry.npmjs.org/npm/-/npm-10.2.3.tgz (sha512-lXdZ7titEN8CH5YJk9C/aYRU9JeDxQ4d8rwIIDsvH3SMjLjHTukB2CFstMiB30zXs4vCrPN2WH6cDq1yHBeJAw==) seems to be corrupted. Trying again.
npm ERR! code EINTEGRITY
npm ERR! sha512-lXdZ7titEN8CH5YJk9C/aYRU9JeDxQ4d8rwIIDsvH3SMjLjHTukB2CFstMiB30zXs4vCrPN2WH6cDq1yHBeJAw== integrity checksum failed when using sha512: wanted sha512-lXdZ7titEN8CH5YJk9C/aYRU9JeDxQ4d8rwIIDsvH3SMjLjHTukB2CFstMiB30zXs4vCrPN2WH6cDq1yHBeJAw== but got sha512-GbUui/rHTl0mW8HhJSn4A0Xg89yCR3I9otgJT1i0z1QBPOVlgbh6rlcUTpHT8Gut9O1SJjWRUU0nEcAymhG2tQ==. (2583937 bytes)

The tests being:

• NpmRule_integTest. succeeds to run npm module using npm_run_ when the package.json file contains local npm (8.5)
• NpmTask_integTest. execute npm command using the npm version specified in the package.json file (8.5)
• NpxTask_integTest. execute npx command using the npm version specified in the package.json file (8.5)

And on Windows, after all of that, the NpmProxy and YarnProxy tests still failed. At least the changes in this commit will allow Windows to build now, right...?

* What went wrong:
Failed to calculate the value of task ':strcalc:compileJava' property
'javaCompiler'.
WindowsRegistry is not supported on this operating system.

Great:

• WindowsRegistry is not supported on this operating system. gradle/native-platform#274 (comment)
• Support ARM64 Windows gradle/gradle#21703

So much for Windows on arm64 for now.

This was obviously quite the learning journey, but it's only reminded me again of how much I hate the Java ecosystem. Yet more days I wish I had back in my life.