VFS

[pepyakin]

It's hard to test the filesystem and NOMT heavily depends on it. However, triggering various conditions on a real filesystem is troublesome. Testing of ENOSPC is concievable, but other conditions that involve partial or torn-writes are less so.

Therefore, I am pretty convinced that we should leverage simulation. In particular, NOMT should depend on the virtual filesystem interface¹. The VFS comes with a real backend, that uses underlying OS, and a completely simulated and controllable filesystem. I envision that the VFS should encompass the responsibilities of IO and file management (creation, file locking, etc). The VFS acts as a contract between the real FS and NOMT, meaning it has some preconditions from and promises to the user. As long as we manage to capture the right preconditions and guarantees, we will be able to test independently that

1. the VFS for maintaining the said guarantees on the real FS
2. as well as NOMT holding to the preconditions mandated by the VFS.

This testing could be performed as both explicit unit tests and fuzzing.

One advantage of this approach is that fuzzing of the surface API as it is done right now takes a lot of time and does something like 4 iterations in second. IMO this is too slow for being useful. Fuzzing against the VFS should improve the pace significantly. However, it's worth reiterating that this makes it very important to get the interface right, because, if we do not distill the right assumptions about the behavior of underlying file systems, then we might be end up testing the wrong thing.

Anoter advantage of this, is that it allows making the OS interface configurable. There were reports of issues coming from io-uring on certain kernel versions. To workaround that, it might be worth using the unix fallback IO implementation, based on a thread pool and sync syscalls, even on systems equipped with io-uring. VFS abstraction might allow just that and I don't think it would be a noticable regression to use dynamic dispatch in this case.

Footnotes

1. the VFS concept originally comes from SQLite. ↩

[rphmeier]

Moved to Linear