Adds --stdout and --stderr args for writing output to files #57

dleehr · 2019-03-13T19:21:01Z

Adds optional command-line arguments to specify files for writing standard output and standard error

If specified, these files are written by piping through tee, meaning that stdout/stderr still are written to the kubernetes pod logs, but also to the named files.
This feature allows other tools that run calrissian in kubernetes (e.g. lando) to declare what files to write to, without a shell and without silencing job logs

Baseline functionality for #17

Python 3 raised ValueError: can't have unbuffered text I/O

…mode - These streams fail to write text if they are binary - Python 3 does not allow unbuffered text output - Instead, just flush the streams at the end of the main function

- The tee process writes to stdout, so this was getting redirected into the stdout file - Fixed by redirecting tee's output to stderr

johnbradley

LGTM

dleehr added 7 commits March 12, 2019 15:21

Adds arguments for user-supplied stdout/stderr files

5e7f742

Add args to example revsort job to specify stdout/stderr files

906a60e

Switch open mode to binary to allow buffering

7c25bc1

Python 3 raised ValueError: can't have unbuffered text I/O

Removes the line that reopens stdout/stderr FDs in unbuffered binary …

0b5334f

…mode - These streams fail to write text if they are binary - Python 3 does not allow unbuffered text output - Instead, just flush the streams at the end of the main function

Fix combining of stdout into stderr

97a703b

- The tee process writes to stdout, so this was getting redirected into the stdout file - Fixed by redirecting tee's output to stderr

use shlex.quote to quote file name in tee shell command

213e561

Tests for stdout/stderr redirection in main

6eaf220

dleehr requested a review from johnbradley March 13, 2019 19:21

johnbradley approved these changes Mar 13, 2019

View reviewed changes

dleehr merged commit 4344785 into master Mar 13, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds --stdout and --stderr args for writing output to files #57

Adds --stdout and --stderr args for writing output to files #57

dleehr commented Mar 13, 2019

johnbradley left a comment

Adds --stdout and --stderr args for writing output to files #57

Adds --stdout and --stderr args for writing output to files #57

Conversation

dleehr commented Mar 13, 2019

johnbradley left a comment

Choose a reason for hiding this comment