[SPARK-4765] Make GC time always shown in UI.

[kayousterhout]

This commit removes the GC time for each task from the set of
optional, additional metrics, and instead always shows it for
each task.

cc @pwendell

[SparkQA]

Test build #24188 has started for PR 3622 at commit 4d1cf4b.

• This patch merges cleanly.

[SparkQA]

Test build #24188 has finished for PR 3622 at commit 4d1cf4b.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24188/
Test FAILed.

[kayousterhout]

MIMA tests pass locally; I rebased this on master to see if that makes the tests pass

[SparkQA]

Test build #24191 has started for PR 3622 at commit e71d893.

• This patch merges cleanly.

[SparkQA]

Test build #24191 has finished for PR 3622 at commit e71d893.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24191/
Test FAILed.

[JoshRosen]

[error]  * method GC_TIME()java.lang.String in object org.apache.spark.ui.jobs.TaskDetailsClassNames does not have a correspondent in new version
[error]    filter with: ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.ui.jobs.TaskDetailsClassNames.GC_TIME")

That class is private, so I think this is a false-positive MiMa warning. You can probably add an exclude and re-run.

[kayousterhout]

@JoshRosen One thing I was wondering about here that I couldn't figure out: what does "private" actually mean in this context? I would have thought it meant that it could only be used in the context of the same file, but that's obviously not the case since it's used in other UI classes.

[JoshRosen]

@kayousterhout Yeah, there must be something about private object where it behaves differently than private class because we ran into the same problem over at #3570: we changed the type of a private object's public field and got a MiMa error.

[SparkQA]

Test build #24255 has started for PR 3622 at commit 15ac242.

• This patch merges cleanly.

[SparkQA]

Test build #24255 has finished for PR 3622 at commit 15ac242.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24255/
Test PASSed.

[kayousterhout]

Ok well suspiciously making the class private[spark] now makes all of the MiMA tests pass...

[ksakellis]

@kayousterhout I ran into this issue with a private case class too on: #3486 and added exclusions. I can try adding private[spark] and see if that fixes the Mima issues too.

Anyways, this patch LGTM

[JoshRosen]

LGTM. Which branches should this be merged into? @pwendell, do you think we should pull this into branch-1.2 since this feature was first introduced there?

[kayousterhout]

Thanks @JoshRosen and @ksakellis! I merged this into master and 1.2 (@pwendell asked me to do this so it could be included in 1.2, to prevent a regression where GC time disappears).