[node-bindings] Nextjs standalone output support

[belgattitude]

Although duckdb-neo works great with Nextjs, when doing a standalone build (automatically chosen when you deploy on lambda, ie vercel), Nextjs isn't able to detect that the node-binding is currently imported and therefore doesn't include the native binary it in the build.

That leads to an error:

⨯ Error: Cannot find module '@duckdb/node-bindings-linux-x64/duckdb.node'
Require stack:
- /var/task/node_modules/@duckdb/node-bindings/duckdb.js
- /var/task/node_modules/@duckdb/node-api/lib/index.js
- /var/task/examples/apps/nextjs-app/.next/server/app/demo/duckdb/page.js
- /var/task/node_modules/next/dist/compiled/next-server/server.runtime.prod.js

To be clear the target is not to make duckdb-neo work on lambda environments (too much limited in ram and cpu) but more to enable standalone builds that reduce considerably the node_modules installation (ie when building docker images...)

Possible culprit

The require located in https://github.com/duckdb/duckdb-node-neo/blob/main/bindings/pkgs/%40duckdb/node-bindings/duckdb.js doesn't inform enough about declared dependencies:

module.exports = require(`@duckdb/node-bindings-${process.platform}-${process.arch}/duckdb.node`);

Which in turns makes nextjs prune the native binary (under the hood nextjs uses https://github.com/vercel/nft) as it isn't able to figure it's actually needed.

Reproduction

I've set up a repository here: https://github.com/belgattitude/flowblade

git clone https://github.com/belgattitude/flowblade.git
cd flowblade
corepack enable
yarn install
cd examples/apps/nextjs-app
yarn build-standalone

Then look at the output in .next/standalone:

cd .next/standalone
ls -la node_modules/@duckdb

You'll see that the native binding wasn't included

Confirmed both locally and on vercel: https://flowblade-next-app.vercel.app/demo/duckdb (it logs the issue)

Next steps

esbuild, sharp... tends to declare all architectures via resolve I think. I'll have a look if I can find a way to make it work

I've checked by changing the require to static

//module.exports = require(`@duckdb/node-bindings-${process.platform}-${process.arch}/duckdb.node`);
module.exports = require(`@duckdb/node-bindings-linux-x64/duckdb.node`);

And it works pretty well

Additional info

To make duckdb-neo works in nextjs / react server components, interesting to keep in mind that this option is needed in next.config.mjs

/** @type {import('next').NextConfig} */
const nextConfig = {
  serverExternalPackages: ['@duckdb/node-api],

[belgattitude]

I've created a pull-request in #127 .

As a current workaround for other users, you can patch the @duckdb/node-bindings with

diff --git a/duckdb.js b/duckdb.js
index 554090737873f748376d134111599ba3b4d8523d..cceca4e517e50d4d91390cf99edd39d3902abc16 100644
--- a/duckdb.js
+++ b/duckdb.js
@@ -1 +1,25 @@
-module.exports = require(`@duckdb/node-bindings-${process.platform}-${process.arch}/duckdb.node`);
+const getRuntimePlatformArch = () => `${process.platform}-${process.arch}`;
+
+let native;
+
+switch(getRuntimePlatformArch()) {
+    case 'linux-x64':
+        native = require('@duckdb/node-bindings-linux-x64/duckdb.node');
+        break;
+    case 'linux-arm64':
+        native = require('@duckdb/node-bindings-linux-arm64/duckdb.node');
+        break;
+    case 'darwin-arm64':
+        native = require('@duckdb/node-bindings-darwin-arm64/duckdb.node');
+        break;
+    case 'darwin-x64':
+        native = require('@duckdb/node-bindings-darwin-x64/duckdb.node');
+        break;
+    case 'win32-x64':
+        native = require('@duckdb/node-bindings-win32-x64/duckdb.node');
+        break;
+    default:
+        throw new Error(`Unsupported platform: ${getRuntimePlatformArch()}`);
+}
+
+module.exports = native;

As an example with yarn you can refer to:

belgattitude/flowblade#334

To check there's a vercel deployment here: https://flowblade-next-app.vercel.app/demo/duckdb

[jraymakers]

Thanks for reporting this! The problem and your proposed fix both make sense. I want to do a bit more research into the alternatives, but it seems likely your PR is a reasonable approach.