[Mesos] expand coarse-grained mode docs #14059
Conversation
|
I've added clarity to the coarse-grained mode documentation, as well as fixed some inconsistencies it had with the new Mesos scheduler |
mgummelt
changed the title
|
Test build #61797 has finished for PR 14059 at commit
|
|
Even for small doc changes it can be useful (to speed up review) to verify that the docs build as expected and mention that under tests. |
|
I added that I've verified the docs render correctly in github |
|
|
||
| The benefit of coarse-grained mode is much lower startup overhead, but | ||
| at the cost of reserving Mesos resources for the complete duration of | ||
| the application. |
There was a problem hiding this comment.
say a word about dynamic allocation?\
|
LGTM except the minor comment. |
| requirements like interactive queries or serving web requests. | ||
| Spark can run over Mesos in two modes: "coarse-grained" (default) and | ||
| "fine-grained". | ||
|
|
|
@rxin ready for merge. Would also like this in the 2.0 branch if possible. |
|
Test build #61868 has finished for PR 14059 at commit
|
|
|
||
| ## Fine-Grained | ||
|
|
||
| In "fine-grained" mode, each Spark task inside the Spark executor runs |
There was a problem hiding this comment.
can you add a message saying this is deprecated as of spark 2.0?
There was a problem hiding this comment.
I just made a separate PR for that: #14078
Does that work?
Also, will this be merged into Spark 2.0.0? I need to make the copy consistent with the version.
|
Maybe we raced. I'm going to merge this one first and merge your deprecation one later. merging in master/2.0. |
## What changes were proposed in this pull request? docs ## How was this patch tested? viewed the docs in github Author: Michael Gummelt <[email protected]> Closes #14059 from mgummelt/coarse-grained. (cherry picked from commit 9c04199) Signed-off-by: Reynold Xin <[email protected]>
## What changes were proposed in this pull request? Documentation changes to indicate that fine-grained mode is now deprecated. No code changes were made, and all fine-grained mode instructions were left in place. We can remove all of that once the deprecation cycle completes (Does Spark have a standard deprecation cycle? One major version?) Blocked on #14059 ## How was this patch tested? Viewed in Github Author: Michael Gummelt <[email protected]> Closes #14078 from mgummelt/deprecate-fine-grained. (cherry picked from commit b1db26a) Signed-off-by: Reynold Xin <[email protected]>
## What changes were proposed in this pull request? Documentation changes to indicate that fine-grained mode is now deprecated. No code changes were made, and all fine-grained mode instructions were left in place. We can remove all of that once the deprecation cycle completes (Does Spark have a standard deprecation cycle? One major version?) Blocked on #14059 ## How was this patch tested? Viewed in Github Author: Michael Gummelt <[email protected]> Closes #14078 from mgummelt/deprecate-fine-grained.
What changes were proposed in this pull request?
docs
How was this patch tested?
viewed the docs in github