Skip to content

Commit

Permalink
SPARK-1990: added compatibility for python 2.6 for ssh_read command
Browse files Browse the repository at this point in the history
https://issues.apache.org/jira/browse/SPARK-1990

There were some posts on the lists that spark-ec2 does not work with Python 2.6. In addition, we should check the Python version at the top of the script and exit if it's too old

Author: Anant <[email protected]>

Closes #941 from anantasty/SPARK-1990 and squashes the following commits:

4ca441d [Anant] Implmented check_optput withinthe module to work with python 2.6
c6ed85c [Anant] added compatibility for python 2.6 for ssh_read command

Conflicts:

	ec2/spark_ec2.py
  • Loading branch information
AtlasPilotPuppy authored and pwendell committed Jun 17, 2014
1 parent d7db2e6 commit 6b5f64a
Showing 1 changed file with 15 additions and 1 deletion.
16 changes: 15 additions & 1 deletion ec2/spark_ec2.py
Original file line number Diff line number Diff line change
Expand Up @@ -642,9 +642,23 @@ def ssh(host, opts, command):
time.sleep(30)
tries = tries + 1

# Backported from Python 2.7 for compatiblity with 2.6 (See SPARK-1990)
def _check_output(*popenargs, **kwargs):
if 'stdout' in kwargs:
raise ValueError('stdout argument not allowed, it will be overridden.')
process = subprocess.Popen(stdout=PIPE, *popenargs, **kwargs)
output, unused_err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
raise subprocess.CalledProcessError(retcode, cmd, output=output)
return output


def ssh_read(host, opts, command):
return subprocess.check_output(
return _check_output(
ssh_command(opts) + ['%s@%s' % (opts.user, host), stringify_command(command)])


Expand Down

0 comments on commit 6b5f64a

Please sign in to comment.