Expand and reorganise webR virtual filesystem API

[georgestagg]

Requires #126.

Expands and reorganises the webR filesystem API,

• Move functions into a FS namespace.
• Add new mkdir, rmdir, and unlink functions.
• Rename existing functions to match Emscripten FS API.
• Re-throw any error payloads returned.

[georgestagg]

There is now quite a lot of repeated code to check for WebRPayloadErr and re-throw the error. We might be able to mitigate the repetition as part of a restructure of WebRPayload into classes.

[georgestagg]

For the moment I've extracted the repeated Error construction into a new webRPayloadError() function.

[lionel-]

I only took a quick look but LGTM.

[gregvolny]

When working on this part, please don't forget : https://github.com/georgestagg/webR/issues/56
Since this request, Persistent Emscripten VFS API is more mature. For example, SQLite is supporting several ways: https://sqlite.org/wasm/doc/trunk/persistence.md
It will also be a must to support this new SQLite Wasm on WebR.
Best,

[georgestagg]

For webR in particular a persistent VFS is tricker than at first glance. We cannot immediately use solutions that work well for other WASM projects because we use a blocking worker thread to maintain our place in the stack while waiting for input for R. Due to this we cannot easily yield to the event loop in the worker thread[1].

From what I currently understand there are several options available, but each have their own drawbacks for us,

• localStorage and sessionStorage are unavailable in web workers, where we perform our R computation.
• "OPFS" looks promising, but at the time of writing is only available in bleeding-edge Chromium browsers.
• Emscripten's new "wasmFS" also looks promising but is currently Work in Progress.
• The IndexedDB API (used by Emscripten's IDBFS driver) is asynchronous only. Since we block in the worker thread we're not able to use this API.
• Service workers can be (ab)used to persist data, but only a single service worker can be active on the page at a time.

I think the future of persistent storage in webR is going to be based around synchronous OPFS or wasmFS API, once they are ready and supported by more browsers. In the meantime, solving #56 would probably involve significant new infrastructure based around passing messages to the main thread to persist data. That amount of work should wait until after the first tagged version of webR has been released.

[1] Asyncify would allow us to yield, but it does not work well for us due to the way the R interpreter works and the large asyncify overhead induced.

[gregvolny]

For webR in particular a persistent VFS is tricker than at first glance. We cannot immediately use solutions that work well for other WASM projects because we use a blocking worker thread to maintain our place in the stack while waiting for input for R. Due to this we cannot easily yield to the event loop in the worker thread[1].

From what I currently understand there are several options available, but each have their own drawbacks for us,

• localStorage and sessionStorage are unavailable in web workers, where we perform our R computation.
• "OPFS" looks promising, but at the time of writing is only available in bleeding-edge Chromium browsers.
• Emscripten's new "wasmFS" also looks promising but is currently Work in Progress.
• The IndexedDB API (used by Emscripten's IDBFS driver) is asynchronous only. Since we block in the worker thread we're not able to use this API.
• Service workers can be (ab)used to persist data, but only a single service worker can be active on the page at a time.

I think the future of persistent storage in webR is going to be based around synchronous OPFS or wasmFS API, once they are ready and supported by more browsers. In the meantime, solving #56 would probably involve significant new infrastructure based around passing messages to the main thread to persist data. That amount of work should wait until after the first tagged version of webR has been released.

[1] Asyncify would allow us to yield, but it does not work well for us due to the way the R interpreter works and the large asyncify overhead induced.

Well understood! Important for us is to have this persistent file storage in the future!