cmd module tab misbehavior

[slcott]

BPO	9033
Nosy	@ronaldoussoren, @ned-deily, @merwok, @zvezdan, @boompig
Files	cmd.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ronaldoussoren'
closed_at = None
created_at = <Date 2010-06-19.15:10:14.122>
labels = ['OS-mac', 'type-bug']
title = 'cmd module tab misbehavior'
updated_at = <Date 2018-01-17.22:24:11.948>
user = 'https://bugs.python.org/slcott'

bugs.python.org fields:

activity = <Date 2018-01-17.22:24:11.948>
actor = 'boompig'
assignee = 'ronaldoussoren'
closed = False
closed_date = None
closer = None
components = ['macOS']
creation = <Date 2010-06-19.15:10:14.122>
creator = 'slcott'
dependencies = []
files = ['17782']
hgrepos = []
issue_num = 9033
keywords = ['patch']
message_count = 13.0
messages = ['108185', '108196', '108200', '108353', '108379', '108380', '108381', '108392', '108792', '109732', '109746', '193157', '310213']
nosy_count = 7.0
nosy_names = ['ronaldoussoren', 'ned.deily', 'eric.araujo', 'zvezdan', 'l0nwlf', 'slcott', 'boompig']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue9033'
versions = ['Python 3.6']

[slcott]

noticed that cmd module does not perform completion using TAB on a macintosh properly. instead, the TAB key just places several blank spaces and moves the cursor forward. what it should do is retrieve a list of possibilities for completing a command.

[merwok]

Thanks for your report. Does the readline module work at all?

2.5 is unsupported now; can you test your code with 2.7, the next stable version?

[l0nwlf]

It seems readline module is not installed on your system.
Quoting Ned Deily's comment from bpo-8365 which will most probably solve your issue:
"bpo-6877 (and subsequent fixes in bpo-8066) allows the Python readline module to be built and linked with the OS X editline (libedit) library rather than with the GNU readline library (which is not included with OS X). However, the libedit included in versions of OS X prior to 10.5 is considered too broken to use here.

By default, if you do not specify an --with-universal-archs other than "32-bit" to configure or if you do not explicitly set MACOSX_DEPLOYMENT_TARGET to another value, configure defaults to using "10.4" (or earlier) so the building of the readline module is skipped. You can check this:

>>> from distutils.sysconfig import get_config_var 
>>> get_config_var('MACOSX_DEPLOYMENT_TARGET')
'10.4'

(Whether this is the best default is another question.)

As it stands, to be able to build the readline module, either:

(1) supply the GNU readline library as a local library, or

(2) ensure you are building with a deployment target of at least 10.5. For example:

./configure MACOSX_DEPLOYMENT_TARGET=10.6 ; make

Also note that option (2) is not available for 3.1.x since the changes to support editline/libedit were not ported to it; they are, however, in 2.6.5, 2.7 (trunk), and 3.2 (py3k)"

[ronaldoussoren]

scott:

• Which OSX version are you using?

• Which Python are you using?

  • What is the value of sys.prefix?
  • How did you install it?

• Does 'import readline' work?

[ronaldoussoren]

Reaction from scot w.r.t. my questions:

os x 10.5.8

python 2.5.1

/System/Library/Frameworks/Python.framework/Versions/2.5

came default with system

i'm going to try activestate python 2.6 and see if that solves the problem.

import readline does work

[ronaldoussoren]

Some notes: The system python on OSX 10.5 and 10.6 is linked to libedit, not GNU readline, and doesn't seem to contain patches that convert stdlib usage of readline APIs to the correct way to bind keystrokes to action with libedit.

This results in failure to use libedit at all.

AFAIK this also affects the generic stdlib when linking libedit, which is supported in 2.6.5, 2.7 and 3.2. Therefore adding more versions.

[ronaldoussoren]

This (untested) patch should fix the issue for the cmd module:

+++ Lib/cmd.py	(working copy)
@@ -112,7 +112,18 @@
                 import readline
                 self.old_completer = readline.get_completer()
                 readline.set_completer(self.complete)
-                readline.parse_and_bind(self.completekey+": complete")
+
+                if 'libedit' in readline.__doc__:
+                    # readline linked to BSD libedit
+                    if self.completekey == 'tab':
+                        key = '^I'
+                    else:
+                        key = self.completekey
+                    readline.parse_and_bind("bind %s rl_complete"%(key,))
+
+                else:
+                    # readline linked to the real readline
+                    readline.parse_and_bind(self.completekey+": complete")
             except ImportError:
                 pass
         try:

[l0nwlf]

Tested the patch and it works for trunk and python3.2 alpha.
Without adding SDK flag with configure readline module failed to build.

Python build finished, but the necessary bits to build these modules were not found:
_gdbm ossaudiodev readline
spwd

However for cmd module, tab completion and history(up|down arrow) worked fine.

[ronaldoussoren]

The attached patch is a workaround for the issue, but isn't someone I'd want to commit without some serious cleanup.

As I noted when support for linking with libedit was merged it would be better to add a translation layer that translates GNU readline configuration lines to BSD libedit ones.

[zvezdan]

Does a translation really need to be in Python?

I use .editrc file in my home directory with this content:

python:bind ^I rl_complete

and everything works fine.

[ronaldoussoren]

We either have to add some translation, or tweak parts of python:

• the cmd module needs to learn how to configure libedit when
  the readline extension was linked to libedit

• the rlcompleter documentation needs to be updated to do the same

And that's just the stdlib. IIRC ipython contains code that uses libedit
style configuration instead of readline when using /usr/bin/python on OSX, and there are probably other.

IMHO the current behavior is confusing: the module is named readline, but sometimes uses libedit behavior. At least I ensured that the usage of libedit can be detected by introspecting readline.__doc__.

A basic translator shouldn't be that hard...

[ronaldoussoren]

I no longer particularly like my patch, although something needs to be done. The easiest way forward is likely a (private) helper function in the readline module that can translate simple readline configuration strings to something that libedit understands and use that in the stdlib where readline bindings are replaced (but of course not for reading ~/.inputrc).

Comparing the libedit <http://linux.die.net/man/5/editrc\> and readline <http://cnswww.cns.cwru.edu/php/chet/readline/readline.html#SEC9\> configuration languages makes is clear that it is not possible to fully translate a readline configuration in a libedit one, but basic conifguration like setting up key-bindings should be easy enough.

[boompig]

I can confirm this behaviour for python 3.6.0 on Mac OS X 10.12.6

[ronaldoussoren]

Issue is still present, although command completion does work in an interactive shell (that is, start cmdloop() from the REPL.

Also: site.py contains code that similar to my "cmd.patch", I intend to create a PR based on it.

[encukou]

This was fixed in #107748, for Python 3.13. From my Mac:

$ PYTHON_BASIC_REPL=1 ./python.exe
Python 3.14.0a0 (heads/main:2c7209a3bdf, Jun 17 2024, 17:10:56) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> i<tab><tab>
id(         if          import      in          input(      int(        is          isinstance( issubclass( iter(      
>>> import readline
>>> readline.<tab><tab>
readline.add_history(                         readline.read_history_file(                  
readline.backend                              readline.read_init_file(                     
readline.clear_history()                      readline.redisplay()                         
[... several lines omitted ...]                  
>>> readline.backend
'editline'

It's a new feature, so it's not backported. But, in Python 3.11 tab completion should work if you install readline and build with ./configure --with-readline=readline.