[SPARK-7305][Streaming][WebUI] Make BatchPage show friendly information when jobs are dropped by SparkListener #5840

zsxwing · 2015-05-01T18:20:15Z

If jobs are dropped by SparkListener, at least we can show the job ids in BatchPage. Screenshot:

…rkListener

zsxwing · 2015-05-01T18:20:22Z

cc @tdas

AmplabJenkins · 2015-05-01T18:22:10Z

Merged build triggered.

AmplabJenkins · 2015-05-01T18:22:19Z

Merged build started.

SparkQA · 2015-05-01T18:23:53Z

Test build #31580 has started for PR 5840 at commit 83dec11.

SparkQA · 2015-05-01T20:02:12Z

Test build #31580 has finished for PR 5840 at commit 83dec11.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- final class Binarizer extends Transformer with HasInputCol with HasOutputCol

AmplabJenkins · 2015-05-01T20:02:16Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-01T20:02:16Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31580/
Test PASSed.

AmplabJenkins · 2015-05-06T17:42:11Z

Merged build triggered.

AmplabJenkins · 2015-05-06T17:42:20Z

Merged build started.

SparkQA · 2015-05-06T17:42:45Z

Test build #31999 has started for PR 5840 at commit 8073b03.

SparkQA · 2015-05-06T17:46:04Z

Test build #31999 has finished for PR 5840 at commit 8073b03.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-06T17:46:05Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-06T17:46:05Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31999/
Test FAILed.

zsxwing · 2015-05-06T18:08:59Z

retest this please.

AmplabJenkins · 2015-05-06T18:12:12Z

Merged build triggered.

AmplabJenkins · 2015-05-06T18:12:20Z

Merged build started.

SparkQA · 2015-05-06T18:13:06Z

Test build #32008 has started for PR 5840 at commit 8073b03.

SparkQA · 2015-05-06T19:12:38Z

Test build #32008 has finished for PR 5840 at commit 8073b03.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-06T19:12:43Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-06T19:12:44Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32008/
Test FAILed.

zsxwing · 2015-05-06T20:03:47Z

retest this please.

AmplabJenkins · 2015-05-06T20:07:12Z

Merged build triggered.

AmplabJenkins · 2015-05-06T20:07:21Z

Merged build started.

SparkQA · 2015-05-06T20:07:55Z

Test build #32020 has started for PR 5840 at commit 8073b03.

SparkQA · 2015-05-06T21:15:37Z

Test build #32020 has finished for PR 5840 at commit 8073b03.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-06T21:15:42Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-06T21:15:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32020/
Test FAILed.

zsxwing · 2015-05-06T22:55:01Z

retest this please.

tdas · 2015-05-07T18:32:01Z

streaming/src/main/scala/org/apache/spark/streaming/ui/BatchPage.scala

+    val lastStageInfo =
+      sparkJobs.flatMap(_.jobUIData).headOption. // Get the first JobUIData
+        // Get the latest Stage info
+        flatMap { sparkJob =>


nit: This is a little weird to read with the continuous line having a line of comment in the middle. took 5 seconds to realize that this is a continuation line.

AmplabJenkins · 2015-05-07T18:32:13Z

Merged build triggered.

AmplabJenkins · 2015-05-07T18:32:22Z

Merged build started.

SparkQA · 2015-05-07T18:34:09Z

Test build #32134 has started for PR 5840 at commit 718765e.

SparkQA · 2015-05-07T18:47:33Z

Test build #32134 has finished for PR 5840 at commit 718765e.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-07T18:47:36Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-07T18:47:37Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32134/
Test FAILed.

AmplabJenkins · 2015-05-07T19:07:13Z

Merged build triggered.

AmplabJenkins · 2015-05-07T19:07:22Z

Merged build started.

SparkQA · 2015-05-07T19:07:59Z

Test build #32137 has started for PR 5840 at commit aca0ba6.

tdas · 2015-05-07T19:09:40Z

LGTM. Will merge when tests pass.

SparkQA · 2015-05-07T20:18:54Z

Test build #32137 has finished for PR 5840 at commit aca0ba6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-07T20:18:59Z

Merged build finished. Test FAILed.

AmplabJenkins · 2015-05-07T20:18:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32137/
Test FAILed.

zsxwing · 2015-05-07T20:23:29Z

retest this please

AmplabJenkins · 2015-05-07T20:27:12Z

Merged build triggered.

AmplabJenkins · 2015-05-07T20:27:18Z

Merged build started.

SparkQA · 2015-05-07T20:29:08Z

Test build #32148 has started for PR 5840 at commit aca0ba6.

SparkQA · 2015-05-07T22:15:06Z

Test build #32148 has finished for PR 5840 at commit aca0ba6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2015-05-07T22:15:11Z

Merged build finished. Test PASSed.

AmplabJenkins · 2015-05-07T22:15:11Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32148/
Test PASSed.

…tion when jobs are dropped by SparkListener If jobs are dropped by SparkListener, at least we can show the job ids in BatchPage. Screenshot: ![b1](https://cloud.githubusercontent.com/assets/1000778/7434968/f19aa784-eff3-11e4-8f86-36a073873574.png) Author: zsxwing <[email protected]> Closes #5840 from zsxwing/SPARK-7305 and squashes the following commits: aca0ba6 [zsxwing] Fix the code style 718765e [zsxwing] Make generateNormalJobRow private 8073b03 [zsxwing] Merge branch 'master' into SPARK-7305 83dec11 [zsxwing] Make BatchPage show friendly information when jobs are dropped by SparkListener (cherry picked from commit 22ab70e) Signed-off-by: Tathagata Das <[email protected]>

…tion when jobs are dropped by SparkListener If jobs are dropped by SparkListener, at least we can show the job ids in BatchPage. Screenshot: ![b1](https://cloud.githubusercontent.com/assets/1000778/7434968/f19aa784-eff3-11e4-8f86-36a073873574.png) Author: zsxwing <[email protected]> Closes apache#5840 from zsxwing/SPARK-7305 and squashes the following commits: aca0ba6 [zsxwing] Fix the code style 718765e [zsxwing] Make generateNormalJobRow private 8073b03 [zsxwing] Merge branch 'master' into SPARK-7305 83dec11 [zsxwing] Make BatchPage show friendly information when jobs are dropped by SparkListener

Make BatchPage show friendly information when jobs are dropped by Spa…

83dec11

…rkListener

Merge branch 'master' into SPARK-7305

8073b03

Make generateNormalJobRow private

718765e

tdas reviewed May 7, 2015
View reviewed changes

Fix the code style

aca0ba6

asfgit closed this in 22ab70e May 8, 2015

zsxwing deleted the SPARK-7305 branch May 8, 2015 00:36

[SPARK-7305][Streaming][WebUI] Make BatchPage show friendly information when jobs are dropped by SparkListener #5840

[SPARK-7305][Streaming][WebUI] Make BatchPage show friendly information when jobs are dropped by SparkListener #5840

Conversation

zsxwing commented May 1, 2015

zsxwing commented May 1, 2015

AmplabJenkins commented May 1, 2015

AmplabJenkins commented May 1, 2015

SparkQA commented May 1, 2015

SparkQA commented May 1, 2015

AmplabJenkins commented May 1, 2015

AmplabJenkins commented May 1, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

SparkQA commented May 6, 2015

SparkQA commented May 6, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

zsxwing commented May 6, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

SparkQA commented May 6, 2015

SparkQA commented May 6, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

zsxwing commented May 6, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

SparkQA commented May 6, 2015

SparkQA commented May 6, 2015

AmplabJenkins commented May 6, 2015

AmplabJenkins commented May 6, 2015

zsxwing commented May 6, 2015

tdas May 7, 2015

Choose a reason for hiding this comment

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

SparkQA commented May 7, 2015

SparkQA commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

SparkQA commented May 7, 2015

tdas commented May 7, 2015

SparkQA commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

zsxwing commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015

SparkQA commented May 7, 2015

SparkQA commented May 7, 2015

AmplabJenkins commented May 7, 2015

AmplabJenkins commented May 7, 2015