Entire FS in a single buffer

[vitonsky]

I need some emscripten FS that would make able to dump content and load content fast, but all operations must happens in RAM, because all data are secret data encrypted on high level.

Ideally it must be a virtual FS that places whole data in single buffer, like tar archive or something.

Is it possible with BrowserFS? Maybe this project have solution for that, or you may navigate me to another solution, or explain how to implement this code myself with BrowserFS?

Ideally the API would looks kinda

// Initializing
const fs = new CustomFS();

// First time we set an empty buffer or maybe create formatted buffer with some util
const content = new Uint8Array().buffer;
fs.setBuffer(content);

// Now custom FS acts with files from our buffer. Read, write, etc.
// We may dump buffer immediately, to snapshot our FS,
// and then load buffer next run, to continue from snapshot state
dumpData(content);

I work on note-taking app that uses PGlite that uses emscripten FS. Users may encrypt their workspace, this is core feature, so i can't just mount NodeFS, since secret data would be placed naked on user disk and may be recovered later. I have issue in PGlite repo with more details if you need a context.

[james-pre]

I think a "raw" backend using a single buffer is a really good idea. I've thought about doing this before, and this is definitely doable.

I've done some research, and I think I'll implement exFAT first, followed by ext4 and btrfs. This won't be quick, but I'm excited to work on it.

Merry Christmas!

[vitonsky]

Sounds good! I would play around zen-fs while you will find time to work on single buffer FS. I can't understand some fundamental concepts, could you please help me.

As i understood, zen-fs focused to work not only in browser, but in any javascript environment, right?

Yesterday i've tried to use zen-fs as storage with InMemo backend for PGlite. They use emscripten FS and as users we can implement custom storage like that: https://github.com/electric-sql/pglite/blob/9aff6739389647ee55e439c7b08a970c40ac3329/packages/pglite/src/fs/nodefs.ts#L24-L28

I tried to attach zen-fs but i can't run it even in tests on my project (i use vitest), i've got unclear errors. Could you explain how to setup zen-fs as emscripten FS on this example?

And happy holidays!

[james-pre]

I tried to attach zen-fs but i can't run it even in tests on my project (i use vitest), i've got unclear errors. Could you explain how to setup zen-fs as emscripten FS on this example?

Sure! In order to avoid any confusion with this issue, please ask via a GitHub discussion or on Discord. Thanks.

[james-pre]

Quick update on this, I'm currently working on a C23 to Typescript compiler and will port the implementations from Linux.