Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: reimplement optimized natural sorting algorithm, speed up ~6 times for case-insensitive sorting #237

Merged
merged 1 commit into from
Oct 2, 2023

Conversation

sxyazi
Copy link
Owner

@sxyazi sxyazi commented Oct 2, 2023

Motivation

This PR fixes the performance regression caused by the introduction of case-insensitive sorting.

Quote from Discord:

I have identified the cause of the performance regression.

It's because the Yazi Git version introduced case-insensitive sorting as the default behavior, and when users set the sorting method to "natural", it uses the natord crate to do it.

The case-insensitive implementation in this crate is very inefficient, and the library hasn't been maintained for 8 years. This has resulted in a doubling of sorting speed in Yazi.

I'm currently working on rewriting a new, optimized sorting algorithm. I'm also considering reverting the default behavior to case-sensitive, since it does have an impact on performance.

Users will only need to manually enable it when they truly require it, so they won't pay the performance cost when it's unnecessary.

BTW, I also stumbled upon this, seems Yazi and exa share the same performance issue 🤣 ogham/exa#367

Benchmark

Sorting 100,000 files with case-insensitive sorting, speeds up by 6 times:

image

Since Yazi didn't introduce any benchmark framework, I used Instant for a rough measurement, which is simple enough. There is benchmark code in natsort.rs as well; you can run it yourself.

@sxyazi sxyazi merged commit 36eb30a into main Oct 2, 2023
@sxyazi sxyazi deleted the pr-9f7ca0a6 branch October 2, 2023 05:17
@sxyazi sxyazi mentioned this pull request Jan 15, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant