url: Cannot load ESM module with newline in file name

[demurgos]

• Version: 10.12.0
• Platform: Linux 64 bit
• Subsystem: Modules

Node fails to load ESM entry points if their filename contains a newline character. This is a regression compared to the current commonjs modules.

The following code fails:

fs.writeFileSync("foo\nbar.mjs", "console.log('Hello, World!');\n");
const spawnRes = cp.spawnSync(process.execPath, ["--experimental-modules", "foo\nbar.mjs"]);
assert(spawnRes.status === 0);
console.error(spawnRew.stderr.toString("UTF-8"));

Expected: no failed assertion, nothing in stderr (appart from the warning that ESM is experimental).
Actual: Failed assertion. Stderr contains:

Error: Cannot find module /tmp/tmp-59206v9LQ3sEjvCA/foobar.mjs
    at search (internal/modules/esm/default_resolve.js:28:12)
    at Loader.resolve [as _resolve] (internal/modules/esm/default_resolve.js:64:11)
    at Loader.resolve (internal/modules/esm/loader.js:58:33)
    at Loader.getModuleJob (internal/modules/esm/loader.js:113:40)
    at Loader.import (internal/modules/esm/loader.js:99:28)
    at asyncESM.loaderPromise.then (internal/modules/cjs/loader.js:734:27)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)
    at startup (internal/bootstrap/node.js:279:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:752:3) code: 'MODULE_NOT_FOUND'

Note that the error message does not even contain the newline in the file name.

For comparison, the following code using commonJS works fine:

fs.writeFileSync("foo\nbar.js", "console.log('Hello, World!');\n");
const spawnRes = cp.spawnSync(process.execPath, ["foo\nbar.js"]);
assert(spawnRes.status === 0);
console.error(spawnRew.stderr.toString("UTF-8"));

[devsnek]

esm uses urls which apparently normalize out newlines, so you'll need to use %0A when you spawn

cp.spawnSync(process.execPath, ["--experimental-modules", "foo%0Abar.mjs"]

[demurgos]

Using your code seems to lead to double-escaping. It still fails. Here is the error message:

Error: Cannot find module /tmp/tmp-59206v9LQ3sEjvCA/foo%250Abar.mjs
...

Furthermore, requiring to escape newlines when spawning a process would not be consistent: am I supplying a relative URL or a system-dependent path? I assume it is a system path. I can successfully pass foo#bar.mjs, foo?bar.mjs, foo bar.mjs (without escaping: as I would for CJS modules).

I am writing a module to convert import.meta.url to the corresponding __filename (file URL to sys-path) and trying a few edge cases. Only newlines cause errors.

[guybedford]

This sounds like a bug to me - the conversion into a URL of the initial path should be /tmp/tmp-59206v9LQ3sEjvCA/foo%0Abar.mjs, which is supposed to be done at https://github.com/nodejs/node/blob/master/lib/internal/modules/cjs/loader.js#L752.

[demurgos]

The bug is in the URL#pathname setter:

const url = new URL("file://");
url.pathname = "/foo\nbar";
console.log(url.toString());
// actual: file:///foobar
// expected: file:///foo%0Abar

[targos]

/cc @nodejs/url

Given:

var url = new URL("http://www.example.com/")
url.pathname = 'foo\nbar';
url.href

• Node and the whatwg-url output 'http://www.example.com/foobar'
• Chrome outputs 'http://www.example.com/foo%0Abar'

[jasnell]

Interesting... yeah Edge, Firefox, and Chrome all produce foo%0Abar for this.

/cc @nodejs/url

[domenic]

If 3/4 browsers all give an answer different from the spec, filing an issue at https://github.com/whatwg/url is a good idea.

That said, in general I'm not sure you should expect arbitrary strings to round-trip through URL parsing and serialization, so there may be a more fundamental issue here. (I haven't read the whole thread.)

[demurgos]

Seems to be related to this line in the spec:

<li><p>Remove all <a>ASCII tab or newline</a> from <var>input</var>.

I confirm that files with a tab in their name also fail. I'll open a issue on the whatwg repo. Chrome and Firefox do not remove tabs and newlines.

Edit: whatwg/url#419

[demurgos]

This bug also occurs with TAB and CR characters. I have a patch that fixes it by percent-encoding these characters before passing them to the pathname setter.

[Trott]

Am I correct to conclude that Node.js is spec-compliant in its treatment of newlines in ES Module paths, although multiple browsers are not? (If so, should this be closed, as the bug is either in the spec or in the browser implementations and not Node.js?)

[guybedford]

@Trott it's a spec concern at this point, yes. But this issue should stay open while the root spec issue at whatwg/url#419 remains unresolved, as which way that goes would determine whether or not this should be a Node bug.