Autoload vim script uses non-specific path to python binary

[kephircheek]

Describe the bug
Following with the official docs

On first run, the plugin creates its own virtualenv using the right Python version and automatically installs Black.

but failed with

Error detected while processing /Users/kephircheek/rebecca/vim/plugged/black/autoload/black.vim:
line  214:
Traceback (most recent call last):
  File "<string>", line 113, in <module>
  File "<string>", line 89, in _initialize_black_env
  File "/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/venv/__init__.py", line 468, in create
    builder.create(env_dir)
  File "/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/venv/__init__.py", line 74, in create
    self.setup_python(context)
  File "/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/venv/__init__.py", line 292, in setup_python
    copier(context.executable, path)
  File "/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/venv/__init__.py", line 235, in symlink_or_copy
    shutil.copyfile(src, dst)
  File "/opt/homebrew/Cellar/[email protected]/3.11.1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/bin/python3'
Error detected while processing function black#Black:

Explanations
My default python3 has version 3.10.9 but python3.11 also installed.
Path '/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/bin/ exists and fit look like

$ ls -1 /opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/bin/
2to3-3.11
idle3.11
pip3
pip3.11
pydoc3.11
python3.11
python3.11-config

There is no python3. bin of other non-default python version also not include python3.
It not seems to be homebrew bug, but may.

Reasons
My version of MacVim uses non-default python version ==3.11 .

$ +cmdline_hist      +langmap           -python            +viminfo
+cmdline_info      +libcall           +python3           +virtualedit
Linking: clang -L. -fstack-protector-strong -L/opt/homebrew/opt/libyaml/lib -L/opt/homebrew/opt/[email protected]/lib -L/opt/homebrew/opt/readline/lib -L. -fstack-protector-strong -L/opt/homebrew/opt/libyaml/lib -L/opt/homebrew/opt/[email protected]/lib -L/opt/homebrew/opt/readline/lib -arch arm64 -L/opt/homebrew/lib -o Vim -lm -lncurses -liconv /opt/homebrew/lib/libintl.a -framework AppKit -L/opt/homebrew/opt/lua/lib -llua5.4 -L/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE -lperl -L/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.11/lib/python3.11/config-3.11-darwin -lpython3.11 -framework CoreFoundation -F/System/Library/Frameworks -framework Tcl -framework CoreFoundation -lruby.3.2 -L/opt/homebrew/Cellar/ruby/3.2.0/lib 

How to fix
python3 hard-coded here.
In my opinion, it should be like f'python{pyver[0]}.{pyver[1]}'.

Environment

• Black's version: stable
• OS and Python version: macOS Monterey (12.6) / Python 3.11.1
• MacVim version: Vi IMproved 9.0 (2022 Jun 28, compiled Sep 15 2022 18:56:17)

[kgaughan]

The root cause of the issue seems to be that unlike 3.10, 3.11 doesn't symlink python3.11 to python3. I think this actually is a Homebrew issue, as previous versions created this symlink.

[kephircheek]

The root cause of the issue seems to be that unlike 3.10, 3.11 doesn't symlink python3.11 to python3.

Yes, it is and it's ok.

I think this actually is a Homebrew issue, as previous versions created this symlink.

Other versions of python, such as 3.8 and 3.9, also have no symlink. The python3 symlink exists only for a default python binary (3.10 for me).
A default python means

$ python3 --version 
Python 3.10.9

The python3 symlink is the next chain:

$ which python3
/opt/homebrew/bin/python3

$ readlink /opt/homebrew/bin/python3
readlink /opt/homebrew/Cellar/[email protected]/3.10.9/bin/python3 

$ readlink /opt/homebrew/Cellar/[email protected]/3.10.9/bin/python3 
../Frameworks/Python.framework/Versions/3.10/bin/python3

$ readlink /opt/homebrew/Cellar/[email protected]/3.10.9/Frameworks/Python.framework/Versions/3.10/bin/python3
python3.10

Since Vim's python should not match a default python, Black should not expect the existence of a python3 symlink.

[kgaughan]

From what I can see in the plugin, autoload/black.vim hasn't ever used any thing but 'python3' as the executable name: https://github.com/psf/black/blame/main/autoload/black.vim#L37

It may be worth updating _get_python_binary to behave more like _get_virtualenv_site_packages and include the major and minor version numbers in the executable name. That, however, doesn't explain why it broke for 3.11 and not previously. My best flailing around is that it might have something to do with 3.11 not being listed in BlackComplete in plugin/black.vim, but I'm just making a wild guess about that, really.

[kephircheek]

From what I can see in the plugin, autoload/black.vim hasn't ever used any thing but 'python3' as the executable name: https://github.com/psf/black/blame/main/autoload/black.vim#L37

It may be worth updating _get_python_binary to behave more like _get_virtualenv_site_packages and include the major and minor version numbers in the executable name.

See #3508

That, however, doesn't explain why it broke for 3.11 and not previously. My best flailing around is that it might have something to do with 3.11 not being listed in BlackComplete in plugin/black.vim, but I'm just making a wild guess about that, really.

Since my Vim is compiled with python3.11 (why not?) and the script autoload/black.vim runs in the Vim environment, the script runs with python3.11. Why does it expect a python3 binary name? It should be python3.11.

In my opinion, the problem is not related to a Python version. If Vim's Python version and symlink python3 are not the same, the autoload will fail. Of course, the problem is just rare and easy to fix locally with missed link creation, but it's still a bug.

[kgaughan]

Then you can close this ticket, as it's covered by that PR.

[scottwn]

I was having this same problem. I pointed my vimrc to @kephircheek's branch and the fix works.