Execution of ! less /etc/passwd hangs in jupyter notebooks

[daritter]

It seems that trying to execute any command which will wait for input will hang in jupyter notebooks.
When using ! to execute a shell command the terminal is set to be an interactive tty. This is a problem for programs like "less" which check if the terminal is a tty and behave accordingly:

! less /etc/passwd

will show the first less screen and keep the kernel in busy state waiting for input while

!! less /etc/passwd

or

os.system("less /etc/passwd")

work as expected.

---

The version of the notebook server is: 5.4.0
The server is running on this version of Python:
Python 3.6.2 (default, Oct 27 2017, 11:20:07) 
[GCC 6.3.0]

Current Kernel Information:
Python 3.6.2 (default, Oct 27 2017, 11:20:07) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

[takluyver]

Yes, this is a known limitation. There's a mismatch between our protocol - which uses an explicit request for input - and terminal interaction, where input can be written at any point.

The environment for running shell commands 'looks like' a terminal because many commands (like ls) use this to decide to produce human-friendly output (e.g. using colours, laying things out in columns). Our output handling has some of the capabilities of a terminal emulator, but not all of them. So whether we decide to 'look like' a terminal or not, there's always something that's not quite right.

[daritter]

Is there a way to identify my program is run by ipython kernel? Some environment variable maybe?
We do have some command line tool which use less for paging and I would like to make those notebook compatible for our users.

[takluyver]

A lot of tools which page their output will respect a PAGER environment variable. You could ensure that os.environ['PAGER'] is set to cat before running them. In fact, it would probably make sense for us to do that in IPython.

[daritter]

Yes, using PAGER is a good idea, thanks. And I would be very happy if IPython would set that variable automatically

[takluyver]

I went to work on that, but discovered that we've been setting PAGER all along - it was added in 2010. b3d10b8

[daritter]

By now I implemented honouring the pager variable from our side for our tools and I can confirm that this works as expected. Seems the problem was mostly on our side, sorry for the noise