Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[CELEBORN-1812] Distinguish sorting-file from sort-tasks waiting to b…
…e submitted ### What changes were proposed in this pull request? Current implementation uses ` shuffleSortTaskDeque.size()` as current sorting file count.This value might be more appropriately described as the sort tasks waiting to be submitted to `fileSorterExecutors`. And the actual current sorting file number ( doing some disk-io operation etc) should be get from `sortingShuffleFiles`. ### Why are the changes needed? Add metrics to monitor current sorting files which is making disk-io operations. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ![image](https://github.com/user-attachments/assets/6ffed37e-ad12-4d8d-a4aa-2b2695a92168) Closes #3040 from Z1Wu/fix/sorting_file_metrics. Authored-by: wuziyi <[email protected]> Signed-off-by: Shuang <[email protected]>
- Loading branch information