Skip to content

Commit

Permalink
more comments
Browse files Browse the repository at this point in the history
  • Loading branch information
cloud-fan committed Feb 11, 2018
1 parent 8dab79a commit 151a92d
Showing 1 changed file with 6 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -99,20 +99,20 @@ public UnsafeKVExternalSorter(
numElementsForSpillThreshold,
canUseRadixSort);
} else {
LongArray pointArray = map.getArray();
// `BytesToBytesMap`'s point array is only guaranteed to hold all the distinct keys, but
// `UnsafeInMemorySorter`'s point array need to hold all the entries. Since `BytesToBytesMap`
// can have duplicated keys, here we need a check to make sure the point array can hold
// all the entries in `BytesToBytesMap`.
final LongArray pointArray;
// The point array will be used to do in-place sort, which require half of the space to be
// The point array will be used to do in-place sort, which requires half of the space to be
// empty. Note: each record in the map takes two entries in the point array, one is record
// pointer, another is the key prefix.
// pointer, another is key prefix. So the required size of point array is `numRecords * 4`.
// TODO: It's possible to change UnsafeInMemorySorter to have multiple entries with same key,
// so that we can always reuse the point array.
if (map.numValues() > map.getArray().size() / 4) {
if (map.numValues() > pointArray.size() / 4) {
// Here we ask the map to allocate memory, so that the memory manager won't ask the map
// to spill, if the memory is not enough.
pointArray = map.allocateArray(map.numValues() * 4L);
} else {
pointArray = map.getArray();
}

// During spilling, the array in map will not be used, so we can borrow that and use it
Expand Down

0 comments on commit 151a92d

Please sign in to comment.