Errors with vite #58
Comments
|
Here’s a reproducible example |
|
I tried using vite with another popular wasm library ( https://www.npmjs.com/package/@sqlite.org/sqlite-wasm ) and it did build and bundle correctly with Anyway, if you have suggestions on how to get EDIT: I’m going to try again tomorrow inside a plain astro component and see if that works any better |
|
This is due to vitejs/vite#7015. Fortunately, there's a workaround for this that seems to work: Details
import { defineConfig } from 'astro/config';
import react from '@astrojs/react';
// https://github.com/vitejs/vite/blob/ec7ee22cf15bed05a6c55693ecbac27cfd615118/packages/vite/src/node/plugins/workerImportMetaUrl.ts#L127-L128
const workerImportMetaUrlRE =
/\bnew\s+(?:Worker|SharedWorker)\s*\(\s*(new\s+URL\s*\(\s*('[^']+'|"[^"]+"|`[^`]+`)\s*,\s*import\.meta\.url\s*\))/g
// https://astro.build/config
export default defineConfig({
integrations: [react()],
server: {
headers: {
'Cross-Origin-Embedder-Policy': 'require-corp',
'Cross-Origin-Opener-Policy': 'same-origin'
},
},
vite: {
build: {
target: 'esnext',
},
optimizeDeps: {
esbuildOptions: {
target: 'esnext'
}
},
worker: {
format: 'es',
// https://github.com/vitejs/vite/issues/7015
// https://github.com/vitejs/vite/issues/14499#issuecomment-1740267849
plugins: [
{
name: 'Disable nested workers',
enforce: 'pre',
transform(code, id) {
if (code.includes('new Worker') && code.includes('new URL') && code.includes('import.meta.url')) {
return code.replace(workerImportMetaUrlRE, `((() => { throw new Error('Nested workers are disabled') })()`);
}
}
}
]
}
}
});However, this would mean that the |
|
Thank you @kleisauke! Unfortunately, I’ve tried disabling the nested plugins, and it broke the dev server. It builds just fine though. Can you describe:
I’m getting that I am using vips in a worker—is that allowed, or does vips spawn it’s own web worker? Or is the way I’m importing the workers the problem? // error
const vipsWorker = new Worker(
new URL("../workers/vips.worker", import.meta.url),
{ type: "module", name: "comlink.worker" },
);
// error
import vipsWorkerUrl from "../workers/vips.worker?worker&url";
const vipsWorker = new Worker(
vipsWorkerUrl,
{ type: "module", name: "comlink.worker" },
);
import type { PluginOption } from "vite";
// https://github.com/vitejs/vite/blob/ec7ee22cf15bed05a6c55693ecbac27cfd615118/packages/vite/src/node/plugins/workerImportMetaUrl.ts#L127-L128
const workerImportMetaUrlRE =
/\bnew\s+(?:Worker|SharedWorker)\s*\(\s*(new\s+URL\s*\(\s*('[^']+'|"[^"]+"|`[^`]+`)\s*,\s*import\.meta\.url\s*\))/g;
// https://github.com/vitejs/vite/issues/7015
// https://github.com/vitejs/vite/issues/14499#issuecomment-1740267849
export const disableNestedWorkers: PluginOption = {
enforce: "pre",
name: "Disable nested workers",
transform(code) {
if (
code.includes("new Worker") &&
code.includes("new URL") &&
code.includes("import.meta.url")
) {
return {
code: code.replaceAll(
workerImportMetaUrlRE,
`((() => { throw new Error('Nested workers are disabled') })()`,
),
// (Disable nested workers plugin) Sourcemap is likely to be incorrect: a plugin (Disable nested workers) was used to transform files, but didn't generate a sourcemap for the transformation. Consult the plugin documentation for help
// - https://github.com/withastro/astro/pull/6817
// - https://github.com/withastro/astro/pull/6817/files#diff-2daffa3917247b6251d31ca5525312790f033402a8d48d6616ec7dcf37b78ef6R86
map: { mappings: "" },
};
}
},
}; |
|
Apologies for the confusion; I mixed it up. You'll need to override the
Using wasm-vips in a worker is allowed. Given that Emscripten's pthreads integration also uses web workers, it can sometimes be tricky to set this up with different module bundlers, as you've noticed. So, for example, if you initiate wasm-vips in a web worker and not overriding the graph TD
A["vips-es6.js #40;bundled#41;"] -- "new Worker#40;new URL#40;'vips-es6.worker.js', import.meta.url#41;, {type: 'module'}#41;" --> vips-es6.worker.js
vips-es6.worker.js -- "import#40;'./vips-es6.js'#41; " --> B["vips-es6.js #40;pre-processed#41;"]
B["vips-es6.js #40;pre-processed#41;"] --> C["throw new Error#40;'Nested workers are disabled'#41;"]
But by overriding that handler, this happens: graph TD
A["vips-es6.js #40;bundled#41;"] -- "new Worker#40;locateFile#40;'vips-es6.worker.js'#41;, {type: 'module'}#41;" --> vips-es6.worker.js
vips-es6.worker.js -- "import#40;'./vips-es6.js'#41; " --> B["vips-es6.js #40;pre-processed#41;"]
B["vips-es6.js #40;pre-processed#41;"] -- "new Worker#40;locateFile#40;'vips-es6.worker.js'#41;, {type: 'module'}#41;" --> vips-es6.worker.js
|
|
I’m really sorry @kleisauke but do you have a full example using vips in a worker? I feel like I’m missing something important. Am I supposed to have a When using But, when I switch to And I need the workers to support ES Modules, so I feel like solving the first error is the way to go. I also tried Please note that I cannot use top-level await with comlink, so I put the My worker file looks like: /// <reference lib="webworker" />
import { expose } from "comlink";
import onetime from "onetime";
// import Vips from "wasm-vips";
// eslint-disable-next-line @typescript-eslint/consistent-type-imports
declare type VipsConstructor = typeof import("wasm-vips");
if (!(typeof importScripts === "function")) {
// determine environment https://stackoverflow.com/a/23619712
throw new TypeError(`[example.worker]: ENVIRONMENT_IS_WORKER is false`);
}
importScripts("./vips.js");
// libvips
// - https://www.npmjs.com/package/wasm-vips
// - https://wasm-vips.kleisauke.nl/playground/
const getVips = onetime(
async () =>
// @ts-expect-error Cannot find name 'Vips'.
// eslint-disable-next-line @typescript-eslint/no-unsafe-call
Vips({
// @ts-expect-error Parameter 'fileName' implicitly has an 'any' type.
locateFile: (fileName, scriptDirectory) =>
`${scriptDirectory}${fileName}`,
mainScriptUrlOrBlob: "./vips.js",
}) as Promise<VipsConstructor>,
);
async function getImageUrl(): Promise<string> {
const vips = await getVips();
// #C83658 as CIELAB triple
const start = [46.479, 58.976, 15.052];
// #D8E74F as CIELAB triple
const stop = [88.12, -23.952, 69.178];
// Makes a lut which is a smooth gradient from start colour to stop colour,
// with start and stop in CIELAB
// let lut = vips.Image.identity() / 255;
let lut = vips.Image.identity().divide(255);
// lut = lut * stop + (1 - lut) * start;
lut = lut.multiply(stop).add(lut.multiply(-1).add(1).multiply(start));
lut = lut.colourspace(vips.Interpretation.srgb /* 'srgb' */, {
source_space: vips.Interpretation.lab, // 'lab'
});
const buffer = await fetch("/assets/images/owl.jpg").then(async (resp) =>
resp.arrayBuffer(),
);
let im = vips.Image.newFromBuffer(buffer);
if (im.hasAlpha()) {
// Separate alpha channel
const withoutAlpha = im.extractBand(0, { n: im.bands - 1 });
const alpha = im.extractBand(im.bands - 1);
im = withoutAlpha
.colourspace(vips.Interpretation.b_w /* 'b-w' */)
.maplut(lut)
.bandjoin(alpha);
} else {
im = im.colourspace(vips.Interpretation.b_w /* 'b-w' */).maplut(lut);
}
// Finally, write the result to a blob
const t0 = performance.now();
const outBuffer = im.writeToBuffer(".jpg");
const t1 = performance.now();
console.log(`Call to writeToBuffer took ${t1 - t0} milliseconds.`);
const blob = new Blob([outBuffer], { type: "image/jpeg" });
const blobURL = URL.createObjectURL(blob);
return blobURL;
}
const workerInterface = {
getImageUrl,
};
expose(workerInterface);
export type ExposedInterface = typeof workerInterface;And I’m importing it in this Astro file: ---
// Usage
// <VipsWorker transition:persist />
---
<script>
import { wrap } from "comlink";
import type { ExposedInterface } from "../workers/vips.worker";
import vipsWorkerUrl from "../workers/vips.worker?worker&url";
// use newer syntax https://v3.vitejs.dev/guide/features.html#import-with-constructors
const vipsWorker = new Worker(
// new URL("../workers/vips.worker", import.meta.url),
vipsWorkerUrl,
{ type: "module", name: "comlink.worker" },
);
const vips = wrap<ExposedInterface>(vipsWorker);
// @ts-expect-error Element implicitly has an 'any' type because type 'typeof globalThis' has no index signature.
globalThis.vips = vips;
console.log("vipsWorker", await vips.getImageUrl());
</script> |
Here's a full example of using wasm-vips in a ES6 web worker with vanilla JavaScript: #15 (comment). I tried to fix your code example above, but it didn't work. I tried this patch: Details--- a/src/workers/vips.worker.ts
+++ b/src/workers/vips.worker.ts
@@ -2,33 +2,18 @@
import { expose } from "comlink";
import onetime from "onetime";
-// import Vips from "wasm-vips";
-
-// eslint-disable-next-line @typescript-eslint/consistent-type-imports
-declare type VipsConstructor = typeof import("wasm-vips");
+import Vips from "wasm-vips";
if (!(typeof importScripts === "function")) {
// determine environment https://stackoverflow.com/a/23619712
throw new TypeError(`[example.worker]: ENVIRONMENT_IS_WORKER is false`);
}
-importScripts("./vips.js");
-
// libvips
// - https://www.npmjs.com/package/wasm-vips
// - https://wasm-vips.kleisauke.nl/playground/
-const getVips = onetime(
- async () =>
- // @ts-expect-error Cannot find name 'Vips'.
- // eslint-disable-next-line @typescript-eslint/no-unsafe-call
- Vips({
- // @ts-expect-error Parameter 'fileName' implicitly has an 'any' type.
- locateFile: (fileName, scriptDirectory) =>
- `${scriptDirectory}${fileName}`,
- mainScriptUrlOrBlob: "./vips.js",
- }) as Promise<VipsConstructor>,
-);
+const getVips = onetime(async () => Vips());
async function getImageUrl(): Promise<string> {
const vips = await getVips();
and modified the "overrides": {
"astro": {
"vite": "^5.0.2"
}
}(this ensures issue vitejs/vite#13367 is not masking issue vitejs/vite#7015) This will hang indefinitely during There's not much I can do for this, please subscribe to vitejs/vite#7015 for updates related to this. |
|
Thank you for trying @kleisauke ! I really appreciate your help and recommendations trying to solve this bug. I’ve followed the issue and look forward to when I can try |
|
Thinking about this further, you might be able to patch Hope this helps. |
|
Thanks @kleisauke! That worked 🎉 I had to find and fix a few paths for the monorepo, but after studying your example with |
|
Actually, @kleisauke, I spoke too soon. I am running into |
|
I was able to get it to work with Is the cors error due to loading the worker as a blob, which causes the origin header to be set to |
Unfortunately,
I think this is indeed the issue. It seems that --- a/astro.config.mjs
+++ b/astro.config.mjs
@@ -10,7 +10,8 @@ export default defineConfig({
},
vite: {
build: {
- target: 'esnext'
+ target: 'esnext',
+ assetsInlineLimit: 0
},
optimizeDeps: {
esbuildOptions: { |
|
@kleisauke disabling all asset inlining has a pretty big performance impact. Is there another way with |
|
I’ve tried some of those rollup options, but I haven’t quite gotten it to work 🤔 |
|
I couldn't get it work with those Rollup options either. Hopefully sometime in the future this can be controlled with the The only two options I can think of to fix this are:
|
|
You’re right, the easiest solution is that |
|
@jlarmstrongiv Which solution did you end up finding? I'd love to check it out. I'm currently trying to get wasm-vips to run in a worker as well, and while #15 (comment) and #58 (comment) are really helpful for trying to understand the problem better, I have yet to get it working on my project (see swissspidy/media-experiments#260) Might be special because I'm using @shopify/web-worker to load code in a blob worker. I need to load wasm-vips in there or from a CDN (using jsdelivr right now) as I otherwise don't have control to add CORS header. Anyway, having more examples of wasm-vips in a web worker would be really helpful to me. Edit: I got it working now! The solution was to load |
|
@swissspidy I tried @shopify/web-worker and it didn’t work for me, presumably because of its webpack and babel dependencies Did you get it working with a vite-based project? |
|
I‘m not using Vite unfortunately, just React in this case. |
|
Commit vitejs/vite@4d1342e introduces a new callback functionality to --- a/astro.config.mjs
+++ b/astro.config.mjs
@@ -10,7 +10,9 @@ export default defineConfig({
},
vite: {
build: {
- target: 'esnext'
+ target: 'esnext',
+ assetsInlineLimit: (filePath) =>
+ filePath.endsWith('.worker.js') ? false : undefined,
},
optimizeDeps: {
esbuildOptions: {I'll close this as "won't fix" for now, as there's not much I can do for this. Please subscribe to vitejs/vite#7015 for updates related to this. |
|
Issue vitejs/vite#7015 is now fixed via PR vitejs/vite#16103, which is included in Vite v5.1.6. This means that the previously mentioned workarounds are no longer needed. 🎉 Here's a minimal working example: |
|
@kleisauke fantastic news 🎉 thank you so much for the update and example |
|
@kleisauke it seems that the vite worker no longer works? It appears that vite is attempting to bundle |
|
I tried the older workaround, but that doesn’t seem to work either. Did something change in vite? I also tried the workaround in vitejs/vite#9879, but those didn’t work either |
|
@jlarmstrongiv This looks like a regression introduced in PR vitejs/vite#15852, I recommend downgrading Vite to v5.1.7 for now. $ npm install [email protected] |
|
Fixed in |
|
Hey @jlarmstrongiv would you mind setting up an example repository using vite and wasm-vips ? I cannot get this to work. |
|
@pepijnolivier the only thing missing from this example is upgrading all dependencies and adding this option to the config: vite: {
optimizeDeps: {
exclude: ["wasm-vips"],
},
}, |
This is my vite plugin, for reference only |
In dev mode
While dev mode starts to work with
This causes errors in build
I would love to use
wasm-vips, I just need to figure out how to import and use itThe text was updated successfully, but these errors were encountered: