PiT 24.7: in GH windows runner flow fails when using node installed in the prepare-frontend step

[manolo]

Description of the bug

When running PiT starters in GH actions for windows, they fail when the node installation needs to be upgraded by flow.
Flow prepare-frontend installs a valid node.exe in ~/.vaadin/node/ but then, it fails when it's used in flow.

Error liness than can be seen in output are:

• prepare-frontend

[INFO] The globally installed Node.js version 18.20.5 is older than the required minimum version 20.0.0. Using Node.js from C:\Users\runneradmin\.vaadin.
[INFO] Couldn't find node.exe. Installing Node and npm to C:\Users\runneradmin\.vaadin.
[INFO] Installing node version v22.12.0
[INFO] Downloading https://nodejs.org/dist/v22.12.0/node-v22.12.0-win-x64.zip to C:\Users\runneradmin\.vaadin\node-v22.12.0-win-x64.zip
[INFO] No proxies configured. If you are behind a proxy server, see https://vaadin.com/docs/latest/configuration/development-mode/node-js#proxy-settings-for-downloading-frontend-toolchain for information on proxy configuration.
[INFO] Unpacking C:\Users\runneradmin\.vaadin\node-v22.12.0-win-x64.zip (34872043 bytes) into C:\Users\runneradmin\.vaadin\node\tmp
[INFO] Copying node binary from C:\Users\runneradmin\.vaadin\node\tmp\node-v22.12.0-win-x64\node.exe to C:\Users\runneradmin\.vaadin\node\node.exe
[INFO] Extracting npm
[INFO] Local node installation successful.

• mvn spring-boot:run

2024-12-30T11:04:39.592Z ERROR 5792 --- [  restartedMain] c.v.f.s.frontend.FrontendToolsLocator    : Failed to read the command '[where, node.exe]' stdout/stderr

...

2024-12-30T11:04:49.667Z ERROR 5792 --- [  restartedMain] c.v.f.s.frontend.FrontendToolsLocator    : Failed to read the command '[C:\Users\runneradmin\.vaadin\node\node.exe, -v]' stdout/stderr

...
Caused by: java.lang.IllegalStateException: Failed to install Node
	at com.vaadin.flow.server.frontend.FrontendTools.installNode(FrontendTools.java:691) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	at com.vaadin.flow.server.frontend.FrontendTools.getNodeExecutable(FrontendTools.java:439) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	at com.vaadin.exampledata.NodeUtil.runScript(NodeUtil.java:36) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.NodeUtil.initialize(NodeUtil.java:29) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.NodeScriptInterface.init(NodeScriptInterface.java:28) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.NodeScriptInterface.getChanceString(NodeScriptInterface.java:34) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.ChanceStringType.getValue(ChanceStringType.java:22) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.ChanceStringType.getValue(ChanceStringType.java:6) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.ExampleDataGenerator.assignValue(ExampleDataGenerator.java:48) ~[exampledata-6.2.0.jar:6.2.0]
	at com.vaadin.exampledata.ExampleDataGenerator.createBean(ExampleDataGenerator.java:38) ~[exampledata-6.2.0.jar:6.2.0]
	... 25 common frames omitted
Caused by: com.vaadin.flow.server.frontend.installer.InstallationException: Unable to detect version of Node. Using command C:\Users\runneradmin\.vaadin\node\node.exe --version
	at com.vaadin.flow.server.frontend.installer.NodeInstaller.getVersion(NodeInstaller.java:574) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	at com.vaadin.flow.server.frontend.installer.NodeInstaller.nodeIsAlreadyInstalled(NodeInstaller.java:238) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	at com.vaadin.flow.server.frontend.installer.NodeInstaller.install(NodeInstaller.java:214) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	at com.vaadin.flow.server.frontend.FrontendTools.installNode(FrontendTools.java:689) ~[flow-server-24.7.0.alpha2.jar:24.7.0.alpha2]
	... 34 common frames omitted

Expected behavior

It installs and uses correctly node, like it happens in linux/mac

Minimal reproducible example

In a github actions workflow select windows-latest runner and execute the followin command sequence

# Clean stuff
rm -rf ~/.m2/repository/
rm -rf ~/.vaadin/node/
# Clone repo
git clone https://github.com/vaadin/designer-tutorial.git
cd designer-tutorial
# Change version
perl -pi -e 's|(\s*<'vaadin.version'>)[^\s]+(</'vaadin.version'>)|${1}24.7.0.alpha1${2}|g' pom.xml
# Run build
mvn -ntp -B

Versions

• Vaadin / Flow version: 24.7.0.alpha1
• OS version: windows-latest in gihub actions

[mcollovati]

The failing execution is not directly from Flow, but from NodeUtil class in exampledata artifact.

[mcollovati]

FrontenUtils.consumeProcessStreams read stdout and stderr asynchronously, using the common ForkJoin pool.
Probaly the common pool size in the container is so small, that the read tasks are not scheduled within 10 seconds (perhaps because of other long running tasks) and the process times out.
Doing the same operation with a dedicated thread pool seems to work fine.
We can maybe consider to add a custom thread pool to be used for Flow internal tasks, instead of the common ForkJoin pool.

[mcollovati]

For the specific issue with the designer-tutorial repository, probably the best option would be to replace the node based data generator with a pure Java solution.

[mcollovati]

After a deeper investingation it looks like that the ForkJoinPool threads (pool size is 3) are currently busy with other long running tasks (e.g. one submitted by DevModeHanlderManagerImpl.initDevModeHandler), and the commands and other commands are postponed by more than 10 seconds.

We should identify all long running tasks submitted to the common pool (e.g. CompletableFuture.runAsync and CompletableFuture.supplyAsync) and use a dedicated executor.

[manolo]

Or might be don't configure any timeout and wait until the pool manages all the tasks?

[manolo]

BTW, one solution might be to avoid using the exampledata library, which has been removed from start.vaadin.com. However, I still think it’s useful for PiT to discover issues like this in a public API. Additionally, customers may have been using the library in their applications if they generated the scaffolding from the start app some time ago.

[mcollovati]

Configuring a timeout would help, but I'm not sure if we can get a suitable value. Currently, it is 10 seconds, and it is not enough in this case. And what if the application code uses CompletableFuture to schedule many tasks at startup? The application will take a long time to start, if the common pool size is small.

I think at least dev-server must have its own pool, to prevent wasting the common pool just when it waits for node tasks to complete. This should be enough to prevent issues like the one described above.