[SPARK-18256] Improve the performance of event log replay in HistoryServer #15756
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrewor14
approved these changes
Nov 3, 2016
| @@ -540,7 +544,8 @@ private[spark] object JsonProtocol { | |||
|
|
|||
| def taskStartFromJson(json: JValue): SparkListenerTaskStart = { | |||
| val stageId = (json \ "Stage ID").extract[Int] | |||
| val stageAttemptId = (json \ "Stage Attempt ID").extractOpt[Int].getOrElse(0) | |||
| val stageAttemptId = | |||
| Utils.jsonOption(json \ "Stage Attempt ID").map(_.extract[Int]).getOrElse(0) | |||
There was a problem hiding this comment.
what's the difference? Is extractOpt really slow?
There was a problem hiding this comment.
if so maybe we should add a scalastyle to ban it
There was a problem hiding this comment.
oh wait you already did 😄
|
LGTM. That's a massive amount of time spent in |
|
Test build #68077 has finished for PR 15756 at commit
|
|
retest this please |
|
Test build #68149 has finished for PR 15756 at commit
|
|
LGTM |
vanzin
approved these changes
Nov 4, 2016
|
Cool. Merging to master! |
uzadude
pushed a commit
to uzadude/spark
that referenced
this pull request
Jan 27, 2017
…erver ## What changes were proposed in this pull request? This patch significantly improves the performance of event log replay in the HistoryServer via two simple changes: - **Don't use `extractOpt`**: it turns out that `json4s`'s `extractOpt` method uses exceptions for control flow, causing huge performance bottlenecks due to the overhead of initializing exceptions. To avoid this overhead, we can simply use our own` Utils.jsonOption` method. This patch replaces all uses of `extractOpt` with `Utils.jsonOption` and adds a style checker rule to ban the use of the slow `extractOpt` method. - **Don't call `Utils.getFormattedClassName` for every event**: the old code called` Utils.getFormattedClassName` dozens of times per replayed event in order to match up class names in events with SparkListener event names. By simply storing the results of these calls in constants rather than recomputing them, we're able to eliminate a huge performance hotspot by removing thousands of expensive `Class.getSimpleName` calls. ## How was this patch tested? Tested by profiling the replay of a long event log using YourKit. For an event log containing 1000+ jobs, each of which had thousands of tasks, the changes in this patch cut the replay time in half:  Prior to this patch's changes, the two slowest methods in log replay were internal exceptions thrown by `Json4S` and calls to `Class.getSimpleName()`:  After this patch, these hotspots are completely eliminated. Author: Josh Rosen <[email protected]> Closes apache#15756 from JoshRosen/speed-up-jsonprotocol.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This patch significantly improves the performance of event log replay in the HistoryServer via two simple changes:
extractOpt: it turns out thatjson4s'sextractOptmethod uses exceptions for control flow, causing huge performance bottlenecks due to the overhead of initializing exceptions. To avoid this overhead, we can simply use our ownUtils.jsonOptionmethod. This patch replaces all uses ofextractOptwithUtils.jsonOptionand adds a style checker rule to ban the use of the slowextractOptmethod.Utils.getFormattedClassNamefor every event: the old code calledUtils.getFormattedClassNamedozens of times per replayed event in order to match up class names in events with SparkListener event names. By simply storing the results of these calls in constants rather than recomputing them, we're able to eliminate a huge performance hotspot by removing thousands of expensiveClass.getSimpleNamecalls.How was this patch tested?
Tested by profiling the replay of a long event log using YourKit. For an event log containing 1000+ jobs, each of which had thousands of tasks, the changes in this patch cut the replay time in half:
Prior to this patch's changes, the two slowest methods in log replay were internal exceptions thrown by
Json4Sand calls toClass.getSimpleName():After this patch, these hotspots are completely eliminated.