Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spammy build output #1280

Closed
chetan opened this issue Sep 25, 2016 · 2 comments
Closed

Spammy build output #1280

chetan opened this issue Sep 25, 2016 · 2 comments

Comments

@chetan
Copy link
Contributor

chetan commented Sep 25, 2016

#1233 considerably improves the output of running hab pkg build ... by quieting the hab pkg install output during studio setup, but once the build actually starts, packages installed by hab-plan-build.sh output progress bars again. I've found the fix is to modify/override the following method:

_install_dependency() {
  if [[ -z "${NO_INSTALL_DEPS:-}" ]]; then
    $HAB_BIN install -u $HAB_DEPOT_URL "$dep" || true
  fi
  return 0
}

from https://github.com/habitat-sh/habitat/blob/master/components/plan-build/bin/hab-plan-build.sh#L616-L621

the relevant line becomes:

    ($HAB_BIN install -u $HAB_DEPOT_URL "$dep" | cat -) || true

This line also needs --verbose removed:

 $_xz_cmd --compress -6 --threads=0 --verbose $tarf

https://github.com/habitat-sh/habitat/blob/master/components/plan-build/bin/hab-plan-build.sh#L2185

While this does work, there should probably be a better way of passing the correct behavior into this script. Perhaps something like wget's --no-verbose?

@smurawski smurawski added this to the Accepted Minor milestone Jan 17, 2017
@smurawski smurawski self-assigned this Jan 17, 2017
@chetan
Copy link
Contributor Author

chetan commented Jan 18, 2017

Did some more digging into this and discovered the following -

  • hab pkg build initially quiets output by requiring isatty on stdin/stdout/stderr
  • however once inside the chrooted env, I observe the following
$ hab pkg build . 2>&1 | cat - # redir/pipe to remove tty

$ ps auxwwwf | grep ..
root     15565  0.1  0.0  25176  3792 /hab/studios/testing/dev/pts/10 Ss+ 17:03   0:00          |               |               \_ /hab/pkgs/core/bash/4.3.42/20161213234235/bin/bash /hab/pkgs/core/hab-plan-build/0.15.0/20161222203825/bin/hab-plan-build .

$ ls -al /proc/15565/fd/
total 0
dr-x------ 2 root root  0 Jan 18 17:04 .
dr-xr-xr-x 9 root root  0 Jan 18 17:03 ..
lrwx------ 1 root root 64 Jan 18 17:04 0 -> /hab/studios/testing/dev/pts/10
lrwx------ 1 root root 64 Jan 18 17:04 1 -> /hab/studios/testing/dev/pts/10
lrwx------ 1 root root 64 Jan 18 17:04 2 -> /hab/studios/testing/dev/pts/10

So although our process initially disabled the tty, once inside the chroot env, we establish a new tty. I think the proper fix here is a probably an environment variable (HAB_VERBOSE=false?) that gets propagated into each subshell as necessary.

fnichol added a commit that referenced this issue Jan 27, 2017
…ariables.

The primary use case driving this feature is running builds in a CI
environment. Some logging streams do not cope well with progress
bar style output which behave by using carriage returns to the terminal
to set its cursor back to the start of the line and "redraw" the current
line. These logging streams frequently start outputing each progress
tick/redraw as a full new line and very quickly consume a large amout of
output. In systems such as Travis there is a upper bound to the number
of log lines provided back to the user. As we care about the build
output not spinners of doom, we needed a way to turn these off.

This change adds a way for users of Habitat tooling (such as `hab`, `hab
pkg build`, `hab studio`, etc.) to explcitly drop single-line progress
bar indicators or to disable text coloring by setting 2 new environment
variables. As this feature is targeted for CI environments, environment
variables were used so that the command issued in CI is virtually
identical to the command issued on a developer's workstation.
Additionally, running a `hab pkg build` will invoke a `hab` process
which invokes a `hab-studio` process which may contain further `hab`
subprocess before completing. The amount of work required to only
support explcit flags on all the related tools would have made this
feature much more complex and error prone. The choice of prefixing these
envrionment variables with `HAB_` ensures that there's almost no chance
these variables would affect the build of any software and so can be
safely passed over the Studio's barrier of isolation.

Currently, `hab`'s  UI system will check 2 things to determine its
behavior:

* Coloring is determined by checking the `TERM` environment variable to
  check the local terminfo database. If the `TERM` supports color, then
  `hab` will add ansi coloring.
* Progress bars is determined by checking if the input and output
  streams (i.e. stdin, stdout, stderr) have a TTY assigned (using
  `libc::isatty()` or equivalent depending in platform).

This change checks an environment variable `HAB_NONINTERACTIVE` which
if set to `"true"`, will override the default detection logic and tell
the UI that it does *not* have a TTY assigned. As a consequence,
downloading and uploading progress bars will be disabled.

Additionally, another environment variable `HAB_NOCOLORING` is checked,
and if set to `"true"` will disable all coloring from `hab`,
`hab-studio`, `hab-plan-build`, etc.

tl;dr
-----

To skip spinners in CI either:

    env HAB_NONINTERACTIVE=true hab pkg build ...

or

    export HAB_NONINTERACTIVE=true
    hab pkg build ...

References #1280

Signed-off-by: Fletcher Nichol <[email protected]>
fnichol added a commit that referenced this issue Jan 27, 2017
…ariables.

The primary use case driving this feature is running builds in a CI
environment. Some logging streams do not cope well with progress bar
style output which behave by using carriage returns to the terminal to
set its cursor back to the start of the line and "redraw" the current
line. These logging streams frequently start outputting each progress
tick/redraw as a full new line and very quickly consume a large amount
of output. In systems such as Travis there is a upper bound to the
number of log lines provided back to the user. As we care about the
build output not spinners of doom, we needed a way to turn these off.

This change adds a way for users of Habitat tooling (such as `hab`, `hab
pkg build`, `hab studio`, etc.) to explicitly drop single-line progress
bar indicators or to disable text coloring by setting 2 new environment
variables. As this feature is targeted for CI environments, environment
variables were used so that the command issued in CI is virtually
identical to the command issued on a developer's workstation.
Additionally, running a `hab pkg build` will invoke a `hab` process
which invokes a `hab-studio` process which may contain further `hab`
subprocess before completing. The amount of work required to only
support explicit flags on all the related tools would have made this
feature much more complex and error prone. The choice of prefixing these
environment variables with `HAB_` ensures that there's almost no chance
these variables would affect the build of any software and so can be
safely passed over the Studio's barrier of isolation.

Currently, `hab`'s  UI system will check 2 things to determine its
behavior:

* Coloring is determined by checking the `TERM` environment variable to
  check the local terminfo database. If the `TERM` supports color, then
  `hab` will add ansi coloring.
* Progress bars is determined by checking if the input and output
  streams (i.e. stdin, stdout, stderr) have a TTY assigned (using
  `libc::isatty()` or equivalent depending in platform).

This change checks an environment variable `HAB_NONINTERACTIVE` which
if set to `"true"`, will override the default detection logic and tell
the UI that it does *not* have a TTY assigned. As a consequence,
downloading and uploading progress bars will be disabled.

Additionally, another environment variable `HAB_NOCOLORING` is checked,
and if set to `"true"` will disable all coloring from `hab`,
`hab-studio`, `hab-plan-build`, etc.

tl;dr
-----

To skip spinners in CI either:

    env HAB_NONINTERACTIVE=true hab pkg build ...

or

    export HAB_NONINTERACTIVE=true
    hab pkg build ...

References #1280

Signed-off-by: Fletcher Nichol <[email protected]>
fnichol added a commit that referenced this issue Jan 28, 2017
…ariables.

The primary use case driving this feature is running builds in a CI
environment. Some logging streams do not cope well with progress bar
style output which behave by using carriage returns to the terminal to
set its cursor back to the start of the line and "redraw" the current
line. These logging streams frequently start outputting each progress
tick/redraw as a full new line and very quickly consume a large amount
of output. In systems such as Travis there is a upper bound to the
number of log lines provided back to the user. As we care about the
build output not spinners of doom, we needed a way to turn these off.

This change adds a way for users of Habitat tooling (such as `hab`, `hab
pkg build`, `hab studio`, etc.) to explicitly drop single-line progress
bar indicators or to disable text coloring by setting 2 new environment
variables. As this feature is targeted for CI environments, environment
variables were used so that the command issued in CI is virtually
identical to the command issued on a developer's workstation.
Additionally, running a `hab pkg build` will invoke a `hab` process
which invokes a `hab-studio` process which may contain further `hab`
subprocess before completing. The amount of work required to only
support explicit flags on all the related tools would have made this
feature much more complex and error prone. The choice of prefixing these
environment variables with `HAB_` ensures that there's almost no chance
these variables would affect the build of any software and so can be
safely passed over the Studio's barrier of isolation.

Currently, `hab`'s  UI system will check 2 things to determine its
behavior:

* Coloring is determined by checking the `TERM` environment variable to
  check the local terminfo database. If the `TERM` supports color, then
  `hab` will add ansi coloring.
* Progress bars is determined by checking if the input and output
  streams (i.e. stdin, stdout, stderr) have a TTY assigned (using
  `libc::isatty()` or equivalent depending in platform).

This change checks an environment variable `HAB_NONINTERACTIVE` which
if set to `"true"`, will override the default detection logic and tell
the UI that it does *not* have a TTY assigned. As a consequence,
downloading and uploading progress bars will be disabled.

Additionally, another environment variable `HAB_NOCOLORING` is checked,
and if set to `"true"` will disable all coloring from `hab`,
`hab-studio`, `hab-plan-build`, etc.

tl;dr
-----

To skip spinners in CI either:

    env HAB_NONINTERACTIVE=true hab pkg build ...

or

    export HAB_NONINTERACTIVE=true
    hab pkg build ...

References #1280

Signed-off-by: Fletcher Nichol <[email protected]>
fnichol added a commit that referenced this issue Jan 31, 2017
…ariables.

The primary use case driving this feature is running builds in a CI
environment. Some logging streams do not cope well with progress bar
style output which behave by using carriage returns to the terminal to
set its cursor back to the start of the line and "redraw" the current
line. These logging streams frequently start outputting each progress
tick/redraw as a full new line and very quickly consume a large amount
of output. In systems such as Travis there is a upper bound to the
number of log lines provided back to the user. As we care about the
build output not spinners of doom, we needed a way to turn these off.

This change adds a way for users of Habitat tooling (such as `hab`, `hab
pkg build`, `hab studio`, etc.) to explicitly drop single-line progress
bar indicators or to disable text coloring by setting 2 new environment
variables. As this feature is targeted for CI environments, environment
variables were used so that the command issued in CI is virtually
identical to the command issued on a developer's workstation.
Additionally, running a `hab pkg build` will invoke a `hab` process
which invokes a `hab-studio` process which may contain further `hab`
subprocess before completing. The amount of work required to only
support explicit flags on all the related tools would have made this
feature much more complex and error prone. The choice of prefixing these
environment variables with `HAB_` ensures that there's almost no chance
these variables would affect the build of any software and so can be
safely passed over the Studio's barrier of isolation.

Currently, `hab`'s  UI system will check 2 things to determine its
behavior:

* Coloring is determined by checking the `TERM` environment variable to
  check the local terminfo database. If the `TERM` supports color, then
  `hab` will add ansi coloring.
* Progress bars is determined by checking if the input and output
  streams (i.e. stdin, stdout, stderr) have a TTY assigned (using
  `libc::isatty()` or equivalent depending in platform).

This change checks an environment variable `HAB_NONINTERACTIVE` which
if set to `"true"`, will override the default detection logic and tell
the UI that it does *not* have a TTY assigned. As a consequence,
downloading and uploading progress bars will be disabled.

Additionally, another environment variable `HAB_NOCOLORING` is checked,
and if set to `"true"` will disable all coloring from `hab`,
`hab-studio`, `hab-plan-build`, etc.

tl;dr
-----

To skip spinners in CI either:

    env HAB_NONINTERACTIVE=true hab pkg build ...

or

    export HAB_NONINTERACTIVE=true
    hab pkg build ...

References #1280

Signed-off-by: Fletcher Nichol <[email protected]>
@smith
Copy link
Contributor

smith commented May 29, 2017

Fixed in #1682. @chetan if this still doesn't do what you want, please reopen. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants