Snippets generated even when not needed

[daxpedda]

The PR #3069 introduced linked modules, link_to!(module = "/test.js"). Currently this is meant to use snippets only if the --split-linked-modules is enabled, but not otherwise.

Currently, snippets are always generated, even in the no-modules target, they can be safely deleted and ignored, but they shouldn't be generated in the first place.

This can be easily tried by using link_to!(module = "/test.js") and trying out the following commands.

• wasm-bindgen --target no-modules
• wasm-bindgen --target web
• wasm-bindgen --target no-modules --split-linked-modules

Happy to give a more comprehensive, easy to use example if desired.

Cc @lukaslihotzki.

[lukaslihotzki]

I understand your issue.

There isn't a clean, small fix for it, because we don't know the target in the function where we could remove the files. Maybe we could disable the write-to-disk of all snippets on no-modules without --split-linked-modules, because snippets cannot be imported modules or split linked modules (as wasm-bindgen would error earlier), so they are embedded linked modules, so their write-to-disk can be skipped. I'll look if that can be a small fix. I wouldn't like to introduce new complexity to fix this.

[RReverser]

link_to can actually help me remove couple of manual hacks from wasm-bindgen-rayon, but only if this behaviour is preserved. Now that it exists, I'm doing something like:

fn _ensure_worker_emitted() {
    // Just ensure that the worker is emitted into the output folder, but don't actually use the URL.
    wasm_bindgen::link_to!(module = "/src/workerHelpers.worker.js");
}

This is because I need the Worker to be emitted alongside the snippet JS so that I can use static construct new Worker(new URL('/src/workerHelpers.worker.js', import.meta.url), { type: 'module' }) that is recognised by bundlers only in this shape.

If snippet wasn't emitted and I could only get a URL value from wasm-bindgen without the statically emitted new Worker(...) wrapper, that would break serveral bundlers.

[RReverser]

Another alternative would be to extend link_to! to allow custom wrappers around it so that devs could do link_to!(module = "...", wrapper = "new Worker({}, {{ type: 'module' }})") and such, but I imagine that would be a bigger change.

[daxpedda]

@RReverser I plan to address this soon and would like to understand your use case a bit more.

Unfortunately I have no clue about bundlers but I'm unsure what your exact requirements are and why you are relying on wasm-bindgen to ship JS files that you aren't even using inside of Rust.

Would you mind going into more detail explaining things from first principle so even I, without any knowledge of the JS ecosystem, can understand the issue and your goal?

[RReverser]

It's used for emitting a Web Worker's entry point alongside the main JS - https://github.com/RReverser/wasm-bindgen-rayon/blob/a947bdce8ef1e4b5456b349bd5b3763fe2516e25/src/lib.rs#L55-L59 - which will be used via new Worker(...) afterwards.

So that emitted file is still used, but shouldn't be imported from the main JS file.

For bundlers, that URL must be statically known name as they will only recognise a pattern like new Worker(new URL("...", import.meta.url)) so I need to ensure that this extra JS is emitted in a statically known location, which is what snippets provide. I wrote about this in an article in the past: https://web.dev/articles/bundling-non-js-resources. Some details have changed, but this pattern is still important.

[daxpedda]

I'm still unsure how this should end up in link_to!() supporting this use case.
link_to!() specifically inlines the linked file into the JS shim without --split-linked-modules, so the generated snippet is unused.

My understanding is that you somehow need a way to for wasm-bindgen to copy your file into the output folder so it can be found by the bundler, but that seems to be a different feature then what link_to!() is intended to provide.

Am I understanding this correctly or am I missing something?

[RReverser]

Yeah that's correct. I know it's a hacky behaviour, but unfortunately the entire support for threads in Wasm is a hack on top of a hack.

My job in this case was/is to maintain a library that at least hides all of that as implementation details and let's users "just use" Rayon.

If there will be a better way to emit JS files into the output directory, I'm happy to switch to a different, dedicated API.

[daxpedda]

Seeing that I'm really unqualified to address this, could you come up with a detailed feature proposal for wasm-bindgen?

[RReverser]

Sorry, I don't think I'll have much time soon :( I general terms though... Any macro similar to current behaviour of link_to! should do.

[daxpedda]

Alright, please let me know if there is something else I can do here.
I will go ahead and merge the corresponding PR.

[RReverser]

So the behaviour above will be broken in the next release of wasm-bindgen? Is there no way to keep it?

[daxpedda]

So the behaviour above will be broken in the next release of wasm-bindgen? Is there no way to keep it?

As I said I'm happy to discuss adding a new feature to wasm-bindgen addressing this problem, but link_to!() should definitely not do this and was creating confusion for users who thought they have to ship the snippet along with the JS shim.