Support mounting local directories in the browser via Filesystem API

[adamziel]

https://developer.mozilla.org/en-US/docs/Web/API/File_System_Access_API would make for a 0 to WordPress dev env in 30 seconds. Also, debugging WordPress in Playground would get much easier.

[adamziel]

Related:

https://github.com/jvilk/BrowserFS
http://jvilk.com/browserfs/1.4.1/

[adamziel]

The File System Access API can read local files and directories, and even exposes a createSyncAccessHandle method for files – super convenient for building Emscripten FS adapter:

// const dirHandle = await window.showDirectoryPicker();
(async function() {
    const fileHandle = await window.showOpenFilePicker();
    const dirHandle = await window.showDirectoryPicker();
    var blob = new Blob([
    `
        self.onmessage = function(e) {
            console.log(e.data);
        };
        // Rest of your worker code goes here.
    `
    ], { type: "text/javascript" })
    
    // Sync methods can only be used in web workers on https sites
    var worker = new Worker(window.URL.createObjectURL(blob));
    worker.onmessage = function(e) {
        console.log("Received: " + e.data);
    }
    worker.postMessage(fileHandle);
    worker.postMessage(dirHandle);
})()

More about synchronous methods. Also, WHATWG Spec.

I also found an Emscripten PR that proposes a EMSCRIPTEN_NATIVE_FS implementation, although it seems to have a few issues.

IDBFS Test

[adamziel]

Exploratory PR.

A mapping of Emscripten concepts to FS API concepts:

Emscripten	FS API	Description
mkdir	FileSystemDirectoryHandle.getDirectory()	Creates a new directory.
symlink		Creates a symbolic link.
rename		Renames a file or directory.
rmdir	FileSystemDirectoryHandle.removeEntry()	Removes an empty directory.
readdir	FileSystemDirectoryHandle.getEntries()	Reads the entries in a directory.
unlink	FileSystemFileHandle.remove()	Deletes a file.
readlink		Reads the target of a symbolic link.
stat	FileSystemHandle.getFile()	Gets metadata of a file.
lstat		Gets metadata of a symbolic link.
chmod		Changes the permissions of a file.
fchmod		Changes the permissions of a file (file descriptor).
chown		Changes the owner of a file.
fchown		Changes the owner of a file (file descriptor).
truncate	stream.truncate()	Truncates or extends a file to a specified length.
ftruncate		Truncates or extends a file (file descriptor).
utime		Changes the access and modification times of a file.
open	FileSystemHandle.getFile()	Opens a file.
createStream	.getWriter()	Creates a stream for writing data.
closeStream	WritableStream.close()	Closes a writable stream.
close	FileSystemSyncAccessHandle.close()	Closes a file after it has been opened.
llseek		Repositions the offset of a file descriptor.
seek	FileSystemWritableStream.seek()
read	Blob.text() or Blob.arrayBuffer()	Reads data from a file.
write	FileSystemWritableFileStream.write()	Writes data to a file.
allocate		Allocates space for a file.
mmap		Maps a file into memory.
msync		Synchronizes a mapped file with the disk.
munmap		Unmaps a mapped file from memory.
ioctl		Controls device parameters of a file.

[adamziel]

Until we can support local filesystem as Emscripten FS, there could be a “refresh” button that brings all the files from local directory into Playground. It could stat them and load just the updated ones. There could be a “update every 1s” checkbox.

it’s similar to the explorations in #19

[adamziel]

Here’s some code from ChatGPT that could make a good starting point:

—-

Write a TypeScript function that uses the Filesystem API in the browser to read a file tree from the local filesystem into a JavaScript object. Each file name should be a key, its contents should be the value, and directories should be represented as nested objects. Parallelize up to 20 filesystem api operations.

type FileTree = {
  [fileName: string]: string | FileTree;
};

async function readFile(file: File): Promise<string> {
  const text = await file.text();
  return text;
}

const MAX_CONCURRENT_OPERATIONS = 20;

async function readDirectory(directory: any): Promise<FileTree> {
  let result: FileTree = {};
  let operations: Promise<any>[] = [];

  for await (const entry of directory.values()) {
    const operation = (async () => {
      if (entry.kind === 'file') {
        const file = await entry.getFile();
        const text = await readFile(file);
        result[entry.name] = text;
      } else if (entry.kind === 'directory') {
        const nestedDirectory = await entry.getDirectory();
        result[entry.name] = await readDirectory(nestedDirectory);
      }
    })();

    operations.push(operation);

    // If we've hit the limit, wait for one to complete before starting a new one
    if (operations.length >= MAX_CONCURRENT_OPERATIONS) {
      await Promise.race(operations).then(() => {
        operations = operations.filter(op => op !== operation);
      });
    }
  }

  // Wait for all operations to complete
  await Promise.all(operations);

  return result;
}


async function readFileTree(rootDirectory: any): Promise<FileTree> {
  const fileTree = await readDirectory(rootDirectory);
  return fileTree;
}

Usage:

async function main() {
  try {
    // Request a handle to the directory from the user
    const rootDirectory = await window.showDirectoryPicker();

    // Read the file tree
    const fileTree = await readFileTree(rootDirectory);

    // Do something with the file tree
    console.log(fileTree);
  } catch (error) {
    console.error('Failed to read file tree:', error);
  }
}

// Call the main function when the user clicks a button, for example
document.getElementById('readFilesButton').addEventListener('click', main);

It could fit the website project in this repo.

[adamziel]

Related Emscripten PR: Add support for read/seek on files in the native file system

[adamziel]

Basic support with manual "sync" button was added in #548