bugfix: Wrong "Duration" in "Active Stages" in stages page

If a stage which has completed once loss parts of data, it will be resubmitted. At this time, it appears that stage.completionTime > stage.submissionTime. Author: shiyun.wxm <[email protected]> Closes #170 from BlackNiuza/duration_problem and squashes the following commits: a86d261 [shiyun.wxm] tow space indent c0d7b24 [shiyun.wxm] change the style 3b072e1 [shiyun.wxm] fix scala style f20701e [shiyun.wxm] bugfix: "Duration" in "Active Stages" in stages page
apache · Mar 19, 2014 · d55ec86 · d55ec86
1 parent a18ea00
commit d55ec86
Showing 1 changed file with 3 additions and 1 deletion.
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala
@@ -106,7 +106,9 @@ private[spark] class StageTable(val stages: Seq[StageInfo], val parent: JobProgr
     val description = listener.stageIdToDescription.get(s.stageId)
       .map(d => <div><em>{d}</em></div><div>{nameLink}</div>).getOrElse(nameLink)
     val finishTime = s.completionTime.getOrElse(System.currentTimeMillis())
-    val duration =  s.submissionTime.map(t => finishTime - t)
+    val duration = s.submissionTime.map { t =>
+      if (finishTime > t) finishTime - t else System.currentTimeMillis - t
+    }
 
     <tr>
       <td>{s.stageId}</td>