--extra-python does not replace Python version by full path on Windows

[hotenov]

Describe the bug

I use pyenv on Windows (pyenv-win) which is not supported by nox (as I noticed :)

But I found two workarounds for my local Nox running with installed Python versions via pyenv.

First, I can pass full paths to installed Pythons in my separate (local) noxfile:

# noxfile.local.py

@nox.session(
    python=[
        r"C:\users\winfan\.pyenv\pyenv-win\versions\3.8.2\python.exe",
        r"C:\users\winfan\.pyenv\pyenv-win\versions\3.9.2\python.exe",
    ],
    reuse_venv=True,
)

and use nox CLI with command nox -f noxfile.local.py. It works, but one more file will need to be maintained (coping from production noxfile.py)

Second, Later I found in docs amazing argument in Nox CLI --extra-python
"That's what I need", I thought. But in my case It does not work it was intended as. At least on Windows.

I tried this command for session with following decorator @nox.session(python=["3.8.2"]):

nox --python 3.8.2 --extra-python "C:\users\winfan\.pyenv\pyenv-win\versions\3.8.2\python.exe"

Nox tells me "Python interpreter 3.8.2 not found". Some of you would say: "Use double backslash \\ power". I tried. The result is the same.

The most interesting thing is that --extra-pythons argument works fine, i.e command:

nox --extra-pythons "C:\users\winfan\.pyenv\pyenv-win\versions\3.8.2\python.exe" "C:\users\winfan\.pyenv\pyenv-win\versions\3.9.2\python.exe"

will run all sessions with Python specified in noxfile.py and with all Pythons passed in command (even without backslash escaping in file path)

What am I doing wrong? Or it's really bug?
I did not find something similar in Open bugs. Should I try to reproduce this issue on Ubuntu with pyenv as well?

My installed packages on Windows 10 20H2 19042.868:

nox 2020.12.31
pyenv 2.64.3

---

How to reproduce

1. Set Nox session with python version installed via pyenv only (no standalone version)
2. Try to "replace" this version executing command nox --python <version> --extra-python "<full_path_to_python>"

Expected behavior

    Nox runs session with specified --extra-python Python interpreter creating separate virtual environment in .nox folder with full path session suffix

[cjolowicz]

Hi @hotenov, thanks for filing this issue!

I believe that what you are describing is expected behavior, because 3.8.2 and C:\users\winfan\.pyenv\pyenv-win\versions\3.8.2\python.exe are not the same interpreter. So you are telling Nox to use interpreter A, while adding an extra interpreter B, which is ignored because you just told it to use A.

Instead you should be passing the same full path to both --python and --extra-python, and it should work. (If I'm not misunderstanding, you already tried this and it worked for you, right?)

IMO it would be nice to have a shorthand --force-python for this (see also #361 (comment)). It does get rather bulky if you need to pass a full path. Would that work for you?

[hotenov]

Hi Claudio! @cjolowicz

Thanks for quick response and amazing Hypermodern articles ;)

I think I understand now, that combination --python and --extra-python is two interpreter A + B
and single --extra-python , i.e. nox --extra-python "full_path_to_python" also works fine.

So, I was confused by example in documentation and word "to replace":

Because I expected that it will be replacing interpreter A with A* on-the-fly
where A* - is local python, i.e. to navigate Nox where to take interpreter for this version.

Can you explain for dummy when such combination (replacement) will work?
if, as you said, interpreter B is ignored if interpreter A is not found, why Nox does not try to run with B and simply stops?

UPD: maybe, in docs should be different versions, for example 3.8.2 and replace with 3.8.10, no?

[cjolowicz]

Let me clarify this with an example.

Suppose your Noxfile has sessions with these interpreters: ["3.8", "3.9"]

If you pass --extra-python 3.10, the sessions will be run with these interpreters instead: ["3.8", "3.9", "3.10"]. In other words, the new interpreter 3.10 is appended to the list of interpreters for each session.

If you pass --python 3.10 --extra-python 3.10, the sessions will be run only with Python 3.10. This is what's meant by "replace" in the documentation.

If you pass only --python 3.10, this will not work because you're selecting an interpreter that is not defined in the Noxfile.

The --python and --extra-python options are completely independent. The --python option selects an interpreter, the --extra-python option adds an interpreter. If you want to change the interpreter used by a session, you need to use both options together. (Hence the idea of having --force-python as a shorthand for passing both options. Currently, Nox does not have this.)

[hotenov]

👍🏻 Excellent examples! I think, you should add them into the docs :) All clear, now. Thanks!

I was hoping that option --python 3.10 --extra-python 3.10 is "manual binding" version and real interpreter path. Now I see that this is not the case.

I have to wait when you add this feature or pyenv-win add correct version detection (and several local versions on Windows). Who is faster?! : )

For now, I close this issue. --extra-pythons is quiet enough for me 😎

[cjolowicz]

Glad this helped.

Have you considered using the official Python binaries on Windows, by the way? Nox should work with them out of the box.

[hotenov]

Yes. I have one :) and Nox picks it up smoothly (with it, I found out that pyenv's installings invisible at all for Nox).
But with pyenv managing Pythons a bit easier. And Nox has a cool workaround. Yes, the terminal command has become a little longer, but this is trivia. One code file is more important :)