[SPARK-4606] Send EOF to child JVM when there's no more data to read. #3460

vanzin · 2014-11-25T20:11:34Z

No description provided.

SparkQA · 2014-11-25T20:15:28Z

Test build #23846 has started for PR 3460 at commit 031207d.

This patch merges cleanly.

SparkQA · 2014-11-25T21:38:00Z

Test build #23846 has finished for PR 3460 at commit 031207d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-25T21:38:04Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23846/
Test PASSed.

JoshRosen · 2014-12-23T23:53:27Z

core/src/main/scala/org/apache/spark/deploy/SparkSubmitDriverBootstrapper.scala

@@ -151,7 +151,8 @@ private[spark] object SparkSubmitDriverBootstrapper {
    val isWindows = Utils.isWindows
    val isSubprocess = sys.env.contains("IS_SUBPROCESS")
    if (!isWindows) {
-      val stdinThread = new RedirectThread(System.in, process.getOutputStream, "redirect stdin")
+      val stdinThread = new RedirectThread(System.in, process.getOutputStream, "redirect stdin",
+        true)


Kind of nitpicky, but I'd probably explicitly name the boolean parameter here (IntelliJ complains otherwise, and besides, it's a nice practice). I can fix this up myself on merge, though.

JoshRosen · 2014-12-23T23:58:39Z

I tried this out and was able to reproduce the original issue based on your JIRA description, and I can confirm that this PR fixes the problem. It looks like this issue also affects 1.1.1, so I'll pull this in there, too.

Therefore, I'm going to merge this into master (1.3.0), branch-1.2 (1.2.1), and branch-1.1 (1.1.2).

Author: Marcelo Vanzin <[email protected]> Closes #3460 from vanzin/SPARK-4606 and squashes the following commits: 031207d [Marcelo Vanzin] [SPARK-4606] Send EOF to child JVM when there's no more data to read. (cherry picked from commit 7e2deb7) Signed-off-by: Josh Rosen <[email protected]>

[SPARK-4606] Send EOF to child JVM when there's no more data to read.

031207d

JoshRosen reviewed Dec 23, 2014
View reviewed changes

asfgit closed this in 7e2deb7 Dec 24, 2014

vanzin deleted the SPARK-4606 branch December 24, 2014 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-4606] Send EOF to child JVM when there's no more data to read. #3460

[SPARK-4606] Send EOF to child JVM when there's no more data to read. #3460

vanzin commented Nov 25, 2014

SparkQA commented Nov 25, 2014

SparkQA commented Nov 25, 2014

AmplabJenkins commented Nov 25, 2014

JoshRosen Dec 23, 2014

JoshRosen commented Dec 23, 2014

[SPARK-4606] Send EOF to child JVM when there's no more data to read. #3460

[SPARK-4606] Send EOF to child JVM when there's no more data to read. #3460

Conversation

vanzin commented Nov 25, 2014

SparkQA commented Nov 25, 2014

SparkQA commented Nov 25, 2014

AmplabJenkins commented Nov 25, 2014

JoshRosen Dec 23, 2014

Choose a reason for hiding this comment

JoshRosen commented Dec 23, 2014