Skip to content

Commit

Permalink
[SPARK-17352][WEBUI] Executor computing time can be negative-number b…
Browse files Browse the repository at this point in the history
…ecause of calculation error

## What changes were proposed in this pull request?

In StagePage, executor-computing-time is calculated but calculation error can occur potentially because it's calculated by subtraction of floating numbers.

Following capture is an example.

<img width="949" alt="capture-timeline" src="https://cloud.githubusercontent.com/assets/4736016/18152359/43f07a28-7030-11e6-8cbd-8e73bf4c4c67.png">

## How was this patch tested?

Manual tests.

Author: Kousuke Saruta <[email protected]>

Closes #14908 from sarutak/SPARK-17352.
  • Loading branch information
sarutak authored and srowen committed Sep 2, 2016
1 parent a3097e2 commit 7ee24da
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
Original file line number Diff line number Diff line change
Expand Up @@ -651,9 +651,9 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") {
}
val executorComputingTime = executorRunTime - shuffleReadTime - shuffleWriteTime
val executorComputingTimeProportion =
(100 - schedulerDelayProportion - shuffleReadTimeProportion -
math.max(100 - schedulerDelayProportion - shuffleReadTimeProportion -
shuffleWriteTimeProportion - serializationTimeProportion -
deserializationTimeProportion - gettingResultTimeProportion)
deserializationTimeProportion - gettingResultTimeProportion, 0)

val schedulerDelayProportionPos = 0
val deserializationTimeProportionPos =
Expand Down

0 comments on commit 7ee24da

Please sign in to comment.