Provide virtual input file system to .build()

[ggoodman]

Description

Allow an initial partial filesystem seed to be provided as an input to a .build() operation.

Use-case

In order to increase the common capabilities between the WASM-based and binary-based implementations, it would be helpful to have some measure of control over the filesystem. I thought that one potential avenue for this that would avoid expensive and complicated coordination between esbuild and the controlling system might be to allow an initial FS snapshot to be seeded.

For my use-case, this would allow me to fully embrace esbuild-wasm and have platform and runtime environment portability.

[evanw]

This is an interesting idea, and is something that I assume would be far faster than a previous proposal you had somewhere for a file system API to essentially intercept readFile and readdir IIRC. I'll think more about this proposal.

[JounQin]

I'm trying to develop an HTML as entry plugin, and I need the virtual file system too, especially for esbuild.serve.

It calls esbuild.build in the onLoad hook of plugin at https://github.com/rx-ts/esbuilder/blob/main/packages/html/src/loader.ts#L93.

When using esbuild.build, there is a physical file system, so that the generated files by esbuild.build in plugin will also be available in the top level HTML files.

However, for esbuild.serve, no virtual file system is provided, and I can't inject the generated files into the service of esbuild.serve. So I have to hack with data:text/javascript;base64,. But I can imagine this will not work when using dynamic import because of multi files.

[aletorrado]

It would be nice to have an output virtual filesystem as well because we do some runtime/dynamic js bundling (esbuild's speed is an incredible fit for that).

[loynoir]

gulp use vinyl as virtual fs
https://gulpjs.com/docs/en/api/vinyl/

[PierBover]

There are many use cases for virtual entries. I've used this feature in Webpack and Rollup many times for creating hydration scripts, dynamic entry points, Svelte components, etc.

It would be great if there was a way to solve this with esbuild!

[jimafisk]

Virtual entrypoints would help me avoid expensive read/writes to the filesystem for Svelte SSR components, which I'm only using to render to HTML and then discard.

[evanw]

Virtual entrypoints would help me avoid expensive read/writes to the filesystem for Svelte SSR components, which I'm only using to render to HTML and then discard.

I've found writing to the file system to actually be more efficient than passing data through esbuild's IPC pipe, especially for large amounts of data. This is why esbuild's transform API deliberately writes the input string to the file system anyway. If your code base is large enough that the performance cost of running esbuild is noticeable, a virtual file system API would probably make things slower instead of faster.

[jakebailey]

For my use-case, this would allow me to fully embrace esbuild-wasm and have platform and runtime environment portability.

FYI, Go 1.21 will feature GOOS=wasip1 GOARCH=wasm which esbuild "just works" under and so should work in WASI runtimes (+/- runtime / Go bugs; the target is new). I already have it working in the browser.

The target still has some issues being worked through for various runtimes, so hopefully by 1.21 it's stable.

[callumlocke]

Just to clarify, is the feature being requested here conceptually similar to @rollup/plugin-virtual or have I misunderstood?

[ggoodman]

I would say that @evanw has proven that a full virtual FS is achievable through https://esbuild.github.io/try/.

A precedent for this was produced in https://github.com/esbuild/esbuild.github.io/blob/main/src/try/fs.ts but what I'd love to see is a bit more of an ergonomic approach / DX. I'd also love to see this sort of thing to:

• Allow plugins to provide non-runtime files that influence the build like package.json and tsconfig.json.

[jakebailey]

That's not really indicitive of the effort needed to do this. The FS API used to achieve that playground page relies on the implementation detail that Go sticks its FS abstractions onto the globals when executed in the browser (https://github.com/golang/go/blob/master/misc/wasm/wasm_exec.js#L16), not that there's been any change internally within esbuild to allow a VFS anywhere other than the browser using this trick. The same goes for my WASI version; it's modifying the host, not esbuild.