Skip to content

Commit

Permalink
Improve performance due to slow indexing
Browse files Browse the repository at this point in the history
Newer versions of tantivy require more memory during the indexing phase.
Otherwise the indexing phase will be a lot slower than in previous
versions.

See quickwit-oss/tantivy#2156 for details.
  • Loading branch information
martinmr committed Sep 18, 2023
1 parent 3641fe8 commit d96851d
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/course_library.rs
Original file line number Diff line number Diff line change
Expand Up @@ -550,8 +550,8 @@ impl LocalCourseLibrary {
reader: None,
};

// Initialize the search index writer with an initial arena size of 50 MB.
let mut index_writer = library.index.writer(50_000_000)?;
// Initialize the search index writer with an initial arena size of 150 MB.
let mut index_writer = library.index.writer(150_000_000)?;

// Convert the list of paths to ignore into absolute paths.
let absolute_root = library_root.canonicalize()?;
Expand Down
4 changes: 4 additions & 0 deletions src/scheduler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -746,11 +746,15 @@ impl DepthFirstScheduler {
if let Some(superseded_by) = superseded_by {
all_superseded_by.extend(superseded_by);
}

let superseded_by = self.data.get_superseded_by(&candidate.course_id);
if let Some(superseded_by) = superseded_by {
all_superseded_by.extend(superseded_by);
}
}
if all_superseded_by.is_empty() {
return candidates;
}

// Filter out the superseding lessons and courses with scores lower than the superseding
// score.
Expand Down

0 comments on commit d96851d

Please sign in to comment.