ESM server builds for node

[jenseng]

As libraries increasingly become ESM-only and other ones do a better job of supporting ESM (e.g. React@18, see here), it would be nice to have an official ESM build option for node-based deployments, e.g. something like serverBuildTarget: "node" (to eventually supplant the "node-cjs" default). This will unlock faster, smaller builds, as the only thing bundled will be the application code; all dependencies (CJS or ESM) will be external and imported natively.

Currently you can use the deprecated serverModuleFormat: "esm" setting (e.g. see here) to get a pure ESM build with no bundled dependencies, but it would be nice to have an official/un-deprecated option.

I've got a branch where I've been playing with this, and it works just fine, e.g. here are some integration tests where we bundle for ESM and it works great despite many dependencies being implemented in CJS (i.e. Remix, React, cjs-only-pkg, etc.).

As far as I can tell, the biggest hurdle is actually around the local dev experience for stacks using remix serve, so we'd need to solve that before making it official. remix serve uses require and resets the require.cache in order to reload the server bundle on each request. If we switch it to import (which works for both ESM and CJS), there is no mechanism to re-import, so currently you'd have to restart the server to see your changes.

Some ideas:

• Automatically restart the server process when the bundle is rebuilt 🤷 . Easiest, but probably the slowest, and browser requests could get a connection refused while it's restarting. Probably not a big deal though.
• Do the above, but have a simple proxy in front of it that queues up requests while the server is restarting. Better UX, but more complex.
• Cache-bust the server bundle with a query string, e.g. await import("build/index.mjs?yolo="+Math.random()). Works, but the big downside is this leaks, as modules never get GCd.
• Roll a custom importer via vm.SourceTextModule. This works, but it also leaks, albeit at about half the rate of the query string approach.
• Import the bundle and possibly handle requests within a Worker. The difficulty here is finding the right boundary that 1. works, and 2. doesn't introduce new restrictions. Payloads must be cloneable, so passing React components (route exports) or arbitrary adapter context (via getLoadContext()) gets problematic.

What do y'all think of the above? And are there other any ESM gotchas I'm missing? Happy to polish up my branch and create a PR once we address any open questions.

[jenseng]

A slightly different approach that could solve the local dev-ex issue would be if esbuild had a new output format specifically for CJS+ESM compatibility. i.e. the idea is to output a CJS build, but rather than having it require any externals, it await imports them. This would allow it to consume external dependencies that use either module format, removing the need for serverDependenciesToBundle. And since the build would still be CJS, we'd still be able to require and reload it via the current mechanism. Top-level await requires node 14.8+, but that seems reasonable since Remix already requires node 14+.

This would really just be a stopgap until Node has a mechanism for re-import-ing something, and it's unclear if esbuild would entertain something like this, but it might be worth exploring if we decide we can't go full ESM just yet (due to the gotchas I listed or others)

[MichaelDeBoey]

#4841 is now merged, which will provide new extra config options (serverConditions, serverMainFields & serverMinify) that can be used to implement this

[jenseng]

Awesome, thanks @MichaelDeBoey!

[jaschaio]

Hey @MichaelDeBoey, I don't find a whole lot of information about serverConditions, serverMainFields and serverMinify. Can you explain how these in combination solve the issue of reloading node_modules when using v2_dev? #7125