wasm-pack's browser and nodejs targets generate different .wasm files (in my project)

[arilotter]

Describe the Bug

wasm-pack's browser and nodejs targets generate different .wasm files in my project (but not in a minimal wasm-pack project)

Steps to Reproduce

1. cd to my repository
2. run the following commands:

# build wasm for browser & nodejs:
wasm-pack build -d build/browser --target browser
wasm-pack build -d build/nodejs --target nodejs

# then convert them to wat
wasm2wat build/browser/*.wasm -o build/browser.wat
wasm2wat build/nodejs/*.wasm -o build/nodejs.wat

diff build/browser.wat build/nodejs.wat

Expected Behavior

I expect the generated .wasm files (and wat files) to be bit-for-bit identical, and the output of wasm-pack to only differ in Javascript bindings.

Actual Behavior

It appears the browser .wasm includes $__wbindgen_realloc and (export "__wbindgen_realloc" (func $__wbindgen_realloc)), while the nodejs .wasm doesn't include either of those.
The diff command outputs

247996,248014d247995
<   (func $__wbindgen_realloc (type 4) (param i32 i32 i32) (result i32)
<     block  ;; label = @1
<       local.get 1
<       i32.const -4
<       i32.gt_u
<       br_if 0 (;@1;)
<       local.get 0
<       local.get 1
<       i32.const 4
<       local.get 2
<       call $__rust_realloc
<       local.tee 1
<       i32.eqz
<       br_if 0 (;@1;)
<       local.get 1
<       return
<     end
<     call $wasm_bindgen::__rt::malloc_failure::h4f0e6944c277bd46
<     unreachable)
250982d250962
<   (export "__wbindgen_realloc" (func $__wbindgen_realloc))

[alexcrichton]

Thanks for the report! This is actually currently intentional behavior in that the target generation can tweak the intrinsics used in the wasm and such, causing different wasms to get produced. For example this is because TextDecoder on node didn't support encodeInto when support for that was first added, but browsers did support it.

Could you expand on why this is a bug for you? It's not currently been intended that we always produce the exact same wasm file, but rather the targets have always been able to tweak the wasm file here and there as necessary for each target.

[arilotter]

From my understanding, a key goal of WebAssembly is portability - a single bytecode format for execution on arbitrary platforms. While the spec does explicitly state that "WebAssembly does not specify any APIs or syscalls, only an import mechanism where the set of available imports is defined by the host environment," I expected Node & Browser Wasm targets to be similar enough that any differences were in the generated JS bindings.
I wanted to use a single .wasm file in the browser, in node, and in other languages through some runtime.
While I understand that this isn't necessarily a goal of wasm-pack, I think that generating the same bytecode when possible for multiple platforms aligns with WebAssembly's goal of portability.

[Pauan]

@arilotter Our plan is to add a new wasm-pack --target which will output two .wasm and .js files, one for the browser and one for Node.

Then all you have to do is specify "main" for Node and "browser" for the browsers, and you will get an npm package which can work in both Node and the browser.

This is a common practice even when writing pure JavaScript packages: developers will often create two versions, one for Node and one for the browser. For example, see canvg.

It's also very common for npm packages to ship both an ES6 module version and a CommonJS module version. This is essentially the npm equivalent of a fat binary.

So since that's the way the npm ecosystem has evolved (multiple versions contained in the same package), that's what we're focusing on.

It also gives us a lot more flexibility, since it's possible to tailor the .wasm and .js for each platform.

I expected Node & Browser Wasm targets to be similar enough that any differences were in the generated JS bindings.

Although Node is based on V8, it often lags far behind the browsers. For example, ES6 modules are already supported by all major browsers, but not in Node.

And I would bet a lot of money that anyref and webidl-bindings will be implemented in all major browsers before it gets implemented in Node. So there's a great example of where we'd want to have different .wasm files for Node and the browsers.

aligns with WebAssembly's goal of portability

WebAssembly's goal of portability only applies to WebAssembly itself, it does not apply to WebAssembly programs (which must import functions from the host, so if the host doesn't have that function then you're out of luck).

Of course having WebAssembly programs which are portable is certainly good, but it's a very difficult goal.

and in other languages through some runtime

That's a lot harder than it sounds at first, since it isn't as simple as just creating some bindings and calling it a day: there are a lot of quirks with how JS works, so any runtime would have to emulate those quirks as well.

For example, the way the event loop works. It's doable, but definitely not easy. You're almost certainly better off creating separate .wasm files for each platform.

Multiple compilation isn't unusual at all: binaries are often compiled only for 64-bit Windows, or only for a specific Linux distro, or only for a specific version of Mac OS, etc.

Even though all those platforms use the x86-64 instruction set, they still need separate versions because of the host APIs and OS quirks. WebAssembly is the equivalent of a CPU instruction set (like x86-64). It doesn't try to smooth over APIs.

Perhaps one day we'll have a truly portable programming API which can be used across all platforms and systems (maybe WASI will be that API), but for now multiple compilation will still be necessary.

[arilotter]

@Pauan Thanks for your detailed comments! I'm doing the "fat binary" equivalent in my code.
Everything you've said makes a lot of sense, and I understand why making portable Wasm binaries would be a hard task with not much gained from accomplishing it.