Skip to content

Commit

Permalink
More docs cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
jessegrosjean committed Feb 8, 2019
1 parent 738d9f6 commit ade977d
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 45 deletions.
44 changes: 21 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,17 @@ Fast recursive directory walk.

- Walk is performed in parallel using rayon
- Results are streamed in sorted order
- Custom sorting/filtering if needed

This crate is inspired by both [`walkdir`](https://crates.io/crates/walkdir)
and [`ignore`](https://crates.io/crates/ignore). It attempts to combine the
parallelism of `ignore` with the streaming iterator based api of `walkdir`.
parallelism of `ignore` with `walkdir`s streaming iterator API.

### Example

Recursively iterate over the "foo" directory sorting by name:

```
```rust
use jwalk::{Sort, WalkDir};

fn main() {
Expand All @@ -25,24 +26,8 @@ fn main() {

### Why would you use this crate?

Performance is the main reason. The following benchmarks walk linux's source
code under various conditions. You can run these benchmarks yourself using
`cargo bench`.

Note in particular that this crate is fast when you want streamed sorted
results. Also note that even when used in single thread mode this crate is
very close to `walkdir` in performance.

This crate's parallelism happens at `fs::read_dir` granularity. If you are
walking many files in a single directory it won't help. On the other hand if
you are walking a hierarchy with many folders and many files then it can
help a lot.

Also note that even though the `ignore` crate has similar performance to
this crate is has much worse latency when you want sorted results. This
crate will start streaming sorted results right away, while with `ignore`
you'll need to wait until the entire walk finishes before you can sort and
start processing the results in sorted order.
Speed is the main reason. The following benchmarks walk linux's source code
under various conditions. Run these benchmarks yourself using `cargo bench`.

| Crate | Options | Time |
|---------|--------------------------------|-----------|
Expand All @@ -59,11 +44,24 @@ start processing the results in sorted order.
| walkdir | sorted | 200.09 ms |
| walkdir | sorted, metadata | 422.74 ms |

Note in particular that this crate is fast when you want streamed sorted
results. Also note that even when used in single thread mode this crate is
very close to `walkdir` in performance.

This crate's parallelism happens at `fs::read_dir` granularity. If you are
walking many files in a single directory it won't help. On the other hand if
you are walking a hierarchy with multiple folders then it can help a lot.

Also note that even though the `ignore` crate has similar performance to
this crate is has much worse latency when you want sorted results. This
crate will start streaming sorted results right away, while with `ignore`
you'll need to wait until the entire walk finishes before you can sort and
start processing the results in sorted order.

### Why wouldn't you use this crate?

Directory traversal is already pretty fast with existing more popular
crates. `walkdir` in particular is very good if you need a strait forward
single threaded solution.
Directory traversal is already pretty fast with existing crates. `walkdir` in
particular is great if you need a strait forward single threaded solution.

This crate processes each `fs::read_dir` as a single unit. Reading all
entries and converting them into its own `DirEntry` representation. This
Expand Down
42 changes: 20 additions & 22 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
//!
//! - Walk is performed in parallel using rayon
//! - Results are streamed in sorted order
//! - Custom sorting/filtering if needed
//!
//! This crate is inspired by both [`walkdir`](https://crates.io/crates/walkdir)
//! and [`ignore`](https://crates.io/crates/ignore). It attempts to combine the
//! parallelism of `ignore` with the streaming iterator based api of `walkdir`.
//! parallelism of `ignore` with `walkdir`s streaming iterator API.
//!
//! # Example
//!
Expand All @@ -25,24 +26,8 @@
//!
//! # Why would you use this crate?
//!
//! Performance is the main reason. The following benchmarks walk linux's source
//! code under various conditions. You can run these benchmarks yourself using
//! `cargo bench`.
//!
//! Note in particular that this crate is fast when you want streamed sorted
//! results. Also note that even when used in single thread mode this crate is
//! very close to `walkdir` in performance.
//!
//! This crate's parallelism happens at `fs::read_dir` granularity. If you are
//! walking many files in a single directory it won't help. On the other hand if
//! you are walking a hierarchy with many folders and many files then it can
//! help a lot.
//!
//! Also note that even though the `ignore` crate has similar performance to
//! this crate is has much worse latency when you want sorted results. This
//! crate will start streaming sorted results right away, while with `ignore`
//! you'll need to wait until the entire walk finishes before you can sort and
//! start processing the results in sorted order.
//! Speed is the main reason. The following benchmarks walk linux's source code
//! under various conditions. Run these benchmarks yourself using `cargo bench`.
//!
//! | Crate | Options | Time |
//! |---------|--------------------------------|-----------|
Expand All @@ -59,11 +44,24 @@
//! | walkdir | sorted | 200.09 ms |
//! | walkdir | sorted, metadata | 422.74 ms |
//!
//! Note in particular that this crate is fast when you want streamed sorted
//! results. Also note that even when used in single thread mode this crate is
//! very close to `walkdir` in performance.
//!
//! This crate's parallelism happens at `fs::read_dir` granularity. If you are
//! walking many files in a single directory it won't help. On the other hand if
//! you are walking a hierarchy with multiple folders then it can help a lot.
//!
//! Also note that even though the `ignore` crate has similar performance to
//! this crate is has much worse latency when you want sorted results. This
//! crate will start streaming sorted results right away, while with `ignore`
//! you'll need to wait until the entire walk finishes before you can sort and
//! start processing the results in sorted order.
//!
//! # Why wouldn't you use this crate?
//!
//! Directory traversal is already pretty fast with existing more popular
//! crates. `walkdir` in particular is very good if you need a strait forward
//! single threaded solution.
//! Directory traversal is already pretty fast with existing crates. `walkdir` in
//! particular is great if you need a single threaded solution.
//!
//! This crate processes each `fs::read_dir` as a single unit. Reading all
//! entries and converting them into its own `DirEntry` representation. This
Expand Down

0 comments on commit ade977d

Please sign in to comment.