Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-7451][YARN] Preemption of executors is counted as failure causing Spark job to fail #5993

Closed
wants to merge 2 commits into from
Closed

[SPARK-7451][YARN] Preemption of executors is counted as failure causing Spark job to fail #5993

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cf8b6cf
Stop counting preemption of executors as failure
May 7, 2015
90900cf
Fix log info message
May 8, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,11 @@ private[yarn] class YarnAllocator(
// Hadoop 2.2.X added a ContainerExitStatus we should switch to use
// there are some exit status' we shouldn't necessarily count against us, but for
// now I think its ok as none of the containers are expected to exit
if (completedContainer.getExitStatus == -103) { // vmem limit exceeded
if (completedContainer.getExitStatus == ContainerExitStatus.PREEMPTED) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this constant exist all the way back to Hadoop 2.2? IIRC preemption was added later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the exit status has it back in 2.1.1-beta

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vanzin yes its been there since 2.1.1-beta. Here is the JIRA which added that constant: https://issues.apache.org/jira/browse/YARN-1049

logInfo("Container killed: " + containerId +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this say "Container preempted?" Also, we shouldn't need to report the diagnostics and exit status because they will always be the same for preemption.

". Exit status: " + completedContainer.getExitStatus +
". Diagnostics: " + completedContainer.getDiagnostics)
} else if (completedContainer.getExitStatus == -103) { // vmem limit exceeded
logWarning(memLimitExceededLogMessage(
completedContainer.getDiagnostics,
VMEM_EXCEEDED_PATTERN))
Expand Down