Use WebAssembly.instantiateStreaming instead of WebAssembly.compileStreaming

[linclark]

Unfortunately there are performance issues with WebAssembly.compileStreaming on iOS. On iOS devices, there is a limited amount of faster memory. Because of this, the engine doesn't know which kind of memory to compile for until instantiate is called.

If I understand correctly, for the JSC engine compileStreaming will just do a memcopy of the .wasm. It's only when instantiate is called that the Memory object is created. Then the engine knows whether it will be using the fast memory, or just a normal malloc'd Memory that needs bounds checks. At this point, it can generate the code appropriately.

Because of this, we plan to recommend that all bundlers use instantiateStreaming instead. Does this cause any issues for webpack?

[sokra]

yes there are multiple issues with that:

---

Pre-knowledge:

webpack tries to treat WASM like ESM.
We try to apply all rules/assumptions that apply to ESM to WASM too.
We assume that in future the WASM JS API may be superseeded by a integration of WASM into the ESM module graph.

This means the imports section in the WASM file is resolved like import statements in ESM and the export section is treated like export in ESM.

WASM modules also have a start section, which is executed when the WASM is instantiated.

In the WASM JS API imports are passed via importsObject to the instantiate method.

---

The ESM spec specifies multiple phases. One phase is the ModuleEvaluation. In this phase all modules are evaulated in the well defined order. This phase is synchronous. All modules evaluate in the same "tick".

When WASM are in the module graph this means:

1. the start section is executed in the same "tick"
2. all dependencies of the WASM are executed in the same "tick"
3. the ESM importing the WASM is executed in the same "tick"

This behavior is not possible when using the Promise-version of instantiate. A Promise always delays it's fulfillment into another "tick".

It's only possible when using a sync version of instantiate (WebAssembly.Instance).

Note: Technically there could be a WASM without start section and without dependencies. In this case this doesn't apply. But we can't assume this is always the case.

---

webpack wants to download and compile the WASM in parallel to downloading the JS code. Using instantiateStreaming won't allow this (when the WASM has dependencies), because instantiate requires passing a importsObject. Creating the importsObject requires evaluating all dependencies/imports of the WASM, so these dependencies need to be downloaded before starting to download the WASM.

When using compileStreaming + new WebAssembly.Instance parallel download and compile is possible, because compileStreaming doesn't require an importsObject. The importsObject can be created when WASM and JS download has finished.

---

I also want to quote the WebAssembly spec. It states that compilation happens in a background thread in compile and instantiation happens on the main thread.

It's not clearly stated but in my optinion JSC's behavior is not spec-comform.

---

Other notes:

WASM also lacks the ability to use live bindings of imported identifiers. Instead the importsObject is spec'ed to be copied. This could have weird problems with circular dependencies and WASM.

It would be great to support getters in the importsObject and the be able to get the exports before the start section is executed.

[TheLarkInn]

cc @jsbastien @flagxor @rossberg I wanted to surface this to you all in hopes maybe we can bring this up for the next WASM CG Meeting if discussions are involved. Primarily around the above concerns.

[jfbastien]

This is worth discussing at the next CG meeting. @TheLarkInn, would you mind adding to the agenda (once it's posted... we just had the meeting today so I haven't created the agenda yet)?

To answer a few points:

1. I think treating WebAssembly like ECMAScript modules is the right thing to do. The JSC implementation uses exactly the same C++ code under the hood, so our behavior should match up very well. Same as ESM, export are handled when the JS object is created (but before execution returns), import is handled at "link" time, and start is called during "evaluate" (after tables and memories have been initialized). That's what I think WebAssembly should standardize as ESM integration.
2. IIUC the main issue you're encountering is that you can't inject yourself between object creation and link, correct? ESM just do it by construction, so even circular imports work.
3. I assume that having the import object as an extra indirection layer is a no-go (especially if you link wasm-wasm with i64)?

Interesting points on the spec. It's not final and I don't think this wording is accurate. It cannot mandate parallelism (unless parallel == 1 is OK). It does not mandate that all compilation be performed (disallowing any future compilation), and cannot either because that would prevent having an interpreter or tiering (so Chakra and SpiderMonkey would then not conform either).

[kmiller68]

@sokra For what it's worth, the original design of Wasm actually differs from what is in the initial spec. See the pull request on the design: WebAssembly/design#1018. I'm not sure where this got lost in the transition to the formal spec but I'll file an issue.

[TheLarkInn]

@kmiller68, so are you saying both methods can be used in paralell?

[lukewagner]

To discourage devs from using the sync WebAssembly.Instance and Module ctors (which, do sync compilation on various engines), Chrome currently throws when these constructors are called with modules bigger than an internal limit (4kb, last I saw). So that, combined with the fact that it's the worst-case load-time behavior for JSC, means sync new Instance should not be used by bundlers.

The ESM spec specifies multiple phases. One phase is the ModuleEvaluation. In this phase all modules are evaulated in the well defined order. This phase is synchronous. All modules evaluate in the same "tick".

I wasn't aware of this spec requirement, but you seem to be quite right!

So given these two constraints, could WebPack:

1. rewrite the .wasm to remove the start section and add the start function as an export
2. call WebAssembly.instantiateStreaming() during the instantiation phase
3. call the previously-start function synchronously during the evaluation phase

To break most cycles involving functions, I would imagine instantiateStreaming being passed shim functions that call the real imported function via a mutable var that gets filled in later.

The main problems with this are:

1. direct .wasm->.wasm imports (and avoiding serializing instantiateStreaming requrests)
2. non-function (i.e., memory, table, global) wasm imports (for which the above shim-calling-a-var trick won't work)

While there are future JS API extensions we've discussed that could address some of these cases, I think it might be ok for a bundler to simply reject these cases (i.e., issue a build error) for now:

• the point of treating a .wasm as an ESM is to use it from JS; wasm-to-wasm linkage is achieved more efficiently via static linking (lld) or LTO; dynamic linking isn't well-supported atm
• as long as dynamic linking isn't well supported, .wasm modules either define their own memory/table or import them directly from a generated glue ESM; there isn't inter-.wasm Memory/Table sharing

However, if these cases are or become important, I think they could be made to work, with some effort; happy to talk more about that.

[sokra]

rewrite the .wasm to remove the start section and add the start function as an export

I also thought about that, but that only works great when WASM has no dependencies.

shim functions
direct .wasm->.wasm imports

Shim function would really defeat future performance improvements in JS engines. I think wasm-to-wasm calls will get optimized in engines soon. The benefit of not going through the JS stack is big (afaik at least for v8). (cc @mstarzinger)

non-function

This would break a lot of good stuff in WASM. No memory sharing, no tables...

---

In my opinion this is not a good way to go. I guess I'll stay with the sync Instance call for now. (Maybe using instantiateStreaming for WASM without start and only function imports from JS.) I hope browsers will allow sync Instance > 4kb. I mean they can still do off-thread compilation in compile and off-thread lazy optimization after instantiation... This works for JS why should it be a problem for WASM.

[lukewagner]

I also thought about that, but that only works great when WASM has no dependencies.

Are there any problems in addition to what is already being discussed?

Shim function would really defeat future performance improvements in JS engines. I think wasm-to-wasm calls will get optimized in engines soon. The benefit of not going through the JS stack is big (afaik at least for v8). (cc @mstarzinger)

When I mentioned shim functions, this was for JS↔wasm imports; and even then they weren't always necessary; just to break cycles. I gave some reasons above why I don't expect wasm↔wasm imports to be common at the bundler stage; have you seen any indications otherwise?

By the time wasm↔wasm imports have become common, we can have increased the expressiveness of JS API (in ways already proposed, e.g., allowing instantiate to accept Promises for imported values) or simply have native ESM-wasm integration.

non-function

This would break a lot of good stuff in WASM. No memory sharing, no tables...

Similarly, until dynamic linking starts being used, every module (or .wasm+.js glue pair) will have its own memory/table, so I don't think we need to worry about multiple .wasm's sharing the same memory/table in the initial support.

That being said, if you wanted to do extra work and support this, it would be possible with another simple .wasm rewrite: convert the module that defines the memory/table to instead import and then create the memory/table in the JS glue code so they can be imported.

---

I guess I'll stay with the sync Instance call for now. (Maybe using instantiateStreaming for WASM without start and only function imports from JS.) I hope browsers will allow sync Instance > 4kb.

Then in practice WebPack wasm integration won't work on Chrome and will have worst-case load-time performance on Safari; that seems pretty unattractive. I think it's important not to let the perfect be the enemy of the good here: we're talking about very simple configurations of .wasm modules for the first few years that could easily and optimally use today's instantiateStreaming.

[jdalton]

@lukewagner The 4 kB limit is a super bummer.
Do you know if that applies to the API exposed in Node too?

[lukewagner]

Sorry, I don't

[jdalton]

For my use case I need the synchronous form for CJS to allow migrating to the asynchronous ESM form. But without the synchronous option I can't really offer the ESM form any time soon since there's no migration path.

The 4 kB limit seems odd since the file is already downloaded, so not blocking for network. It's just CPU crunching which could be done in something like a web worker.

Update:

Grabbing some larger wasm payloads, 15 kB and 41 kB, I was able to get output from:

WebAssembly.Module.exports(new WebAssembly.Module(fs.readFileSync("./large.wasm")))

Which I'm guessing means there is no restriction there (a good thing).

[rossberg]

@jdalton, a web worker only helps if it can run concurrently, which necessitates an asynchronous API.

Still, not everybody is happy with the 4kB limit for synchronous compilation. Web platform folks felt very strongly about it, though. I don't actually know if V8 activates it on Node as well.

[jdalton]

A web worker can run the synchronous Module call off the UI thread. I think the 4 kB limit is a step too far. We get it, synchronous isn't ideal, but it has a place. Putting a blocker in it like this is stop-energy that hinders adoption and worse fosters bad feels among those folks with the ecosystem impact to encourage or discourage further adoption.

[rossberg]

What's the point of off-loading to another thread when the UI thread has to block on that other thread finishing?

Agreed with the rest.

[jdalton]

What's the point of off-loading to another thread when the UI thread has to block on that other thread finishing?

I'm not going to bikeshed various web worker implementations. You get the gist, it's gravy when you can offload to another thread for crunching.

Agreed with the rest.

🙌

[lukewagner]

@jdalton Just confirmed also; I guess it's a browser-only limitation.

[sokra]

When I mentioned shim functions, this was for JS↔wasm imports; and even then they weren't always necessary; just to break cycles.

cycles are not the problem in this case (they are another problem).

Consider a part of you application is loaded on demand and contains WASM (this is always the case with webpack when using WASM). Assuming the WASM module depends on some other JS module in that part of the application.

When the user requests that part of the application I want to start downloading the JS and the WASM. I also want to prepare the WASM to be used. This could happen in parallel.

Now the problem with instanciateStreaming. You need to pass all imports to instanciateStreaming. But these imports do come from the JS module which should be loaded in parallel. I could solve this with shim functions, but I always need them.

Using compileStreaming + sync Instance make parallel download and compilation possible.

I gave some reasons above why I don't expect wasm↔wasm imports to be common at the bundler stage; have you seen any indications otherwise?

I my opinion this it not uncommon. I imagine the WASM future as many small WASM modules interleaved with JS modules. WASM to WASM is possible and great, because you could pass complex binary objects between them instead of converting to JS values. Many WASM modules can share a memory WASM module (which gives you alloc and free) and pass pointer around. WASM-to-WASM is great and we shouldn't limit or prevent it.

as long as dynamic linking isn't well supported

Everything for dynamic linking with imports and exports are in place. Any performance issues left with this could be easily solved. In my option this is the way to go.

I favor dynamic linking over static linking because:

• Code Splitting works great with ti
• Static linking can be an optimization on top of the dependency graph and dynamic linking

Dynamic linking should be the way you author WASM.

Are there any problems in addition to what is already being discussed?

You may use WASM from CommonJS, conditionally:

if(Math.random() < 0.5)
  require("./something.wasm");

This must execute sync. In this case webpack calls compileStreaming while chunk load and new Instance when the module is actually used.

Are there any problems in addition to what is already being discussed?

---

My conclusion:

The ESM spec requires sync evaluation, that's only possible with new WASM.Instance, while keeping the full WASM feature set.

Either the ESM spec is changed to allow async evaluation, which is don't recommend.

Or the WASM implementations are changed to allow and support sync instantiate. The WASM spec doesn't have to be changed, it already allows sync instantiate.

Or we give up on the approach to make WASM like ESM.

[rossberg]

@sokra:

Now the problem with instanciateStreaming. You need to pass all imports to instanciateStreaming. But these imports do come from the JS module which should be loaded in parallel. I could solve this with shim functions, but I always need them.

We have discussed generalising the instantiate function to allow its import object to be a collection of promises of individual objects (or even allow the individual entities to be promises). That should solve the parallel compilation use case without requiring shims.

[lukewagner]

Ah, ok, I was imagining the JS file that called instantiateStreaming also contained all the JS code (it had already been downloaded), but it sounds like these are in separate files, so I see why function shims would always be needed.

Now the problem with instanciateStreaming. You need to pass all imports to instanciateStreaming.
But these imports do come from the JS module which should be loaded in parallel. I could solve this > with shim functions, but I always need them.

Agreed with @rossberg here, using shims as a polyfill.

I my opinion this it not uncommon. I imagine the WASM future as many small WASM modules interleaved with JS modules. WASM to WASM is possible and great, because you could pass complex binary objects between them instead of converting to JS values. Many WASM modules can share a memory WASM module (which gives you alloc and free) and pass pointer around.

It's not possible to take two arbitrary wasm modules and have them share memory and tables; they have to be specifically compiled (probably with the same toolchain for the same language) to work this way. This is because these wasm modules must carefully coordinate use of linear memory and table index space to avoid clobbering each other and also coordinate and basic impl details like malloc/free. Right now, there is not a non-experimental toolchain that does this that I know of.

Moreover, even when this is possible, I don't think it will be common for multiple wasm ESMs to share memory because it is, I think, a quite good thing for unrelated modules to have separate memories/tables: otherwise there will be subtle corruption bugs that only surface when particular modules are used together, leaks will be hard to track down, and OOMs on 32-bit devices will be more likely due to larger memories. It will probably be a good idea in the future to use dynamic linking to factor out common runtime code (e.g., libc, libm), but in that case it is only the Module being shared between Instances, not the Memory or Table.

Everything for dynamic linking with imports and exports are in place.

Not in the toolchain: it may be possible at the bundler level but, right now, there is not a non-experimental toolchain that will actually emit dynamically-linked wasm code.

---

Why can't we go with the shims? I think they'll work fine w/ very little overhead (probably zero overhead once the Promise-taking instantiateStreaming is present).

[TheLarkInn]

Ignoring multi lang modules compiled to wasm, I don't see why a narrowerer scope of single lang compiled WASM modules sharing memory being a bad idea.

…

On Mon, Feb 19, 2018, 1:16 PM Luke Wagner ***@***.***> wrote: Ah, ok, I was imagining the JS file that called instantiateStreaming also contained all the JS code (it had already been downloaded), but it sounds like these are in separate files, so I see why function shims would always be needed. Now the problem with instanciateStreaming. You need to pass all imports to instanciateStreaming. But these imports do come from the JS module which should be loaded in parallel. I could solve this > with shim functions, but I always need them. Agreed with @rossberg <https://github.com/rossberg> here, using shims as a polyfill. I my opinion this it not uncommon. I imagine the WASM future as many small WASM modules interleaved with JS modules. WASM to WASM is possible and great, because you could pass complex binary objects between them instead of converting to JS values. Many WASM modules can share a memory WASM module (which gives you alloc and free) and pass pointer around. It's not possible to take two arbitrary wasm modules and have them share memory and tables; they have to be specifically compiled (probably with the same toolchain for the same language) to work this way. This is because these wasm modules must carefully coordinate use of linear memory and table index space to avoid clobbering each other and also coordinate and basic impl details like malloc/free. Right now, there is not a non-experimental toolchain that does this that I know of. Moreover, even when this is possible, I don't think it will be common for multiple wasm ESMs to share memory because it is, I think, a quite good thing for unrelated modules to have separate memories/tables: otherwise there will be subtle corruption bugs that only surface when particular modules are used together, leaks will be hard to track down, and OOMs on 32-bit devices will be more likely due to larger memories. It will probably be a good idea in the future to use dynamic linking to factor out common runtime code (e.g., libc, libm), but in that case it is only the Module being shared between Instances, not the Memory or Table. Everything for dynamic linking with imports and exports are in place. Not in the toolchain: it may be possible at the bundler level but, right now, there is not a non-experimental toolchain that will actually emit dynamically-linked wasm code. ------------------------------ Why can't we go with the shims? I think they'll work fine w/ very little overhead (probably zero overhead once the Promise-taking instantiateStreaming is present). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6433 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADQBMPV7ecdBl1w7wld8zOKPdiazKgIqks5tWeSXgaJpZM4R2Qni> .

[lukewagner]

Practically speaking, dynamic linking of unrelated modules (i.e., modules that aren't compiled all together as part of the same program/app/system) requires a stable source-language ABI which doesn't exist for any of compile-to-wasm language I've seen (and probably shouldn't until ABI-affecting wasm features like multiple return values and exception handling land). So I think that's the entire story for the next 1-2 years unless something totally new pops up.

But even after that, I think modules should default to being standalone and defining their own memory. The benefits are basically the same as those for having a microkernel OS architecture:

• if there is a crash/leak/corruption, it's easy to diagnose who is at fault
• encourages JS-friendly API using types that don't require sharing memory (like streams, strings, things that copy) instead of low-level APIs that require the JS client to immediately worry about "which address space is this i32 relative to? do I have ownership of this memory and need to free it (and how)?"

We can definitely hypothesize situations where it would be more efficient for modules to share memory, but there are a couple different ways to go about fixing that and it's not clear that a brute-force type of dynamic linking is the right way. It's be useful to have the use cases present to help guide us through this design space since it's inherently in optimization territory.

[sokra]

Ok I created an extra challenge.

Here is a test case which I believe is impossible to solve without sync Instance.

https://github.com/webpack/webpack/pull/6529/files

The dependency graph looks like this:

a -> tracker
  -> b -> tracker
  -> wasm -> tracker
          -> c

Which results in the following evaluation order according to the ESM: tracker, b, c, wasm, a (all in the same tick)

Feel free to "compile" the test case anyway you like and try using instanciateStreaming. Even when Promises as imports would be allowed, it won't work in my opinion.

But maybe you surprise me with a clever approach and I was wrong.

[mathiasbynens]

FWIW, using WebAssembly.instantiateStreaming where applicable matches Google’s official recommendation on how to load WebAssembly modules efficiently.

[jdalton]

@mathiasbynens I think that's great for general consumption, however I believe the root issue here is that webpack itself is using synchronous APIs in a more responsible manner (in workers?) and is only being blocked by an arbitrary 4kB limit imposed by Chrome browser specifically. (It's been a while so things might have shifted)

Related: #6475 (comment).

[jfbastien]

@jdalton AFAIK Chrome's 4kB limits on sync compilation should only be in effect for non-main-thread compilation. If that's not the case then it's a bug in Chrome.

[gahaas]

@jdalton Synchronous compilation should work in Chrome 66 on workers. There was indeed an issue but it got fixed.

[TheLarkInn]

Interesting... @sokra spinning into a worker would mix a few features that we can/want to land.

…

On Fri, Apr 13, 2018, 9:27 AM gahaas ***@***.***> wrote: Synchronous compilation should work in Chrome 66 on workers. There was indeed an issue but it got fixed. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6433 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADQBMEQoz00APeilDbQy5CX4WebROt5Tks5toNH1gaJpZM4R2Qni> .

[gahaas]

@jdalton I mixed up issues. For me, already in Chrome 65 new WebAssembly.Module works in workers without size limit on the input. If you can send me an example where it does not work, I will investigate that issue.

FYI, I mixed it up with WebAssembly.compiledStreaming, which is only supported in Chrome 66 on workers.

[mathiasbynens]

@jfbastien

AFAIK Chrome's 4kB limits on sync compilation should only be in effect for non-main-thread compilation.

The opposite is true: the 4 KB size limits only apply on the main thread. There are no limits in workers. https://cs.chromium.org/chromium/src/third_party/blink/renderer/bindings/core/v8/v8_initializer.cc?l=437 (I guess you meant s/non-//?)

[jfbastien]

Yes.

[sokra]

This is fixed. We now use instantiateStreaming when possible. Falling back to compileStreaming + async instantiate for WASM-to-WASM dependency chains.

Falling back non-streaming variants when streaming is not supported by the browser.

[sokra]

Note: adding Promise support to the importsObject would allow us to remove the compileStreaming + async instantiate fallback.

[cars10]

Can someone post an example about how to correctly import wasm now?

A simple

import * as wasm from 'hello_world.gc.wasm'

does not seem to work.

[xtuc]

@cars10 do you have an error?