Rimraf version 4, first pass

[isaacs]

Remaining todo:

• bin script
• take an array of paths and rimraf them all
• benchmark against system rm -rf subprocess, rimraf v3, and native, but in a way that doesn't just pick a winner at random 😬
• Drop support for node v10, or handle the lack of fs.promises
• Provide custom fs functions? Idk, does anyone really need that feature? Dropping it for now. It's easy to bring back if it's an issue, but if no one requests it, better to not have the added complexity.

[coveralls]

Coverage increased (+27.0%) to 100.0% when pulling 71067e7 on v4 into 8c10fb8 on main.

[coveralls]

Coverage increased (+27.0%) to 100.0% when pulling 71067e7 on v4 into 8c10fb8 on main.

[coveralls]

Coverage increased (+27.0%) to 100.0% when pulling 71067e7 on v4 into 8c10fb8 on main.

[isaacs]

So.... the move-then-remove approach is about 2x to 10x slower on Windows (for all of the shallow-broad, deep-narrow, and mid-mid folder tree structures I tested, whether the dirs contained a mix of folders and files, all files, or all folders). Also, I can't get the EBUSY/ENOTEMPTY errors to trigger using the posix approach on Windows. (The good news is that the posix approach is comparably performant as the native approach, on macos linux and windows, so that's validating.)

So I think the way forward here is:

• add EBUSY/ENOTEMPTY/EMFILE/ENFILE exponential backoff
• when encountering EBUSY on Windows, check if the folder is process.cwd(), and if so, process.chdir(dirname(path)) to get out of it.
• just have one algorithm used for both posix and windows, and export rimraf.native() and rimraf.manual()
• use directory-streaming when available (the better to delete very large directories with)

[isaacs]

Reference to the way that Microsoft's own tools remove directories: https://github.com/microsoft/BuildXL/blob/3508b0d6c4dea5ad53cdeba53819a2b3024e67e5/Public/Src/Utilities/Native/IO/Windows/FileUtilities.Win.cs#L165

[damingerdai]

Is there any progress on this pr?

[albertodiazdorado]

I'd love to have this thing released. Not having any dependencies sounds very good and it's part of the PR :D

[isaacs]

Well, quite a long time coming, eh?

Refactored the windows strategy so that it does a normal naive recursive rm, but first deleting all files, then all folders, and falling back to move-then-remove only when it encounters an error indicating that a file is taking too long to remove from a directory. (Doing move-then-remove in all cases is wayyyyy too slow.)

Also, the windows approach uses exponential backoff (or merely retries in the sync case) to handle EBUSY, ENFILE, and EMFILE errors.

This mirrors the strategy used by the Microsoft brand tools that I could find source code for. In the simple async case, performance is comparable to the posix recursive rm (meaning, better than Node's builtin for the async case, and only slightly worse for the sync case).

Only thing lacking is tests for retry-busy.js, and I should probably add a .d.ts file for it.

[isaacs]

It looks like I can't kick off the benchmark workflow until it's merged in, but seems to be showing favorable results locally on windows and macOS, so I'm gonna land it.