Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize mark segments as unused #15352

Merged
merged 1 commit into from
Nov 9, 2023
Merged

Optimize mark segments as unused #15352

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions server/src/main/java/org/apache/druid/metadata/SqlSegmentsMetadataQuery.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,10 @@ public CloseableIterator<DataSegment> retrieveUnusedSegments(
/**
* Marks the provided segments as either used or unused.
*
* For better performance, please try to
* 1) ensure that the caller passes only used segments to this method when marking them as unused.
* 2) Similarly, please try to call this method only on unused segments when marking segments as used with this method.
*
* Returns the number of segments actually modified.
*/
public int markSegments(final Collection<SegmentId> segmentIds, final boolean used)
Expand Down Expand Up @@ -176,7 +180,7 @@ public int markSegments(final Collection<SegmentId> segmentIds, final boolean us
}

/**
* Marks all segments for a datasource unused that are *fully contained by* a particular interval.
* Marks all used segments that are *fully contained by* a particular interval as unused.
*
* Returns the number of segments actually modified.
*/
Expand All @@ -186,7 +190,8 @@ public int markSegmentsUnused(final String dataSource, final Interval interval)
return handle
.createStatement(
StringUtils.format(
"UPDATE %s SET used=:used, used_status_last_updated = :used_status_last_updated WHERE dataSource = :dataSource",
"UPDATE %s SET used=:used, used_status_last_updated = :used_status_last_updated "
+ "WHERE dataSource = :dataSource AND used = true",
dbTables.getSegmentsTable()
)
)
Expand All @@ -202,7 +207,8 @@ public int markSegmentsUnused(final String dataSource, final Interval interval)
return handle
.createStatement(
StringUtils.format(
"UPDATE %s SET used=:used, used_status_last_updated = :used_status_last_updated WHERE dataSource = :dataSource AND %s",
"UPDATE %s SET used=:used, used_status_last_updated = :used_status_last_updated "
+ "WHERE dataSource = :dataSource AND used = true AND %s",
dbTables.getSegmentsTable(),
IntervalMode.CONTAINS.makeSqlCondition(connector.getQuoteString(), ":start", ":end")
)
Expand Down