Substitutions for virtualenv-related sections

[stanislavlevin]

Since tox-3.8.0 the substituted virtualenv-paths (like {envpython} or {envsitepackagesdir}) have become relative.

For example, let use a simple tox configuration like:

[testenv]
commands = echo {envpython} {envsitepackagesdir}

First, check against tox-3.7.0:

python run-test-pre: PYTHONHASHSEED='4105452275'
python runtests: commands[0] | echo /usr/src/RPM/BUILD/test/.tox/python/bin/python /usr/src/RPM/BUILD/test/.tox/python/lib/python3/site-packages
/usr/src/RPM/BUILD/test/.tox/python/bin/python /usr/src/RPM/BUILD/test/.tox/python/lib/python3/site-packages
  /usr/src/RPM/BUILD/test$ /bin/echo /usr/src/RPM/BUILD/test/.tox/python/bin/python /usr/src/RPM/BUILD/test/.tox/python/lib/python3/site-packages

And compare this with tox-3.8.0+:

python run-test-pre: PYTHONHASHSEED='299716092'
python runtests: commands[0] | echo /usr/src/RPM/BUILD/test/.tox/python/bin/python /usr/src/RPM/BUILD/test/.tox/python/lib/python3/site-packages
[19930] /usr/src/RPM/BUILD/test$ /bin/echo .tox/python/bin/python .tox/python/lib/python3/site-packages
.tox/python/bin/python .tox/python/lib/python3/site-packages

So, the virtualenv subst paths became relative instead of absolute ones.

The documentation says nothing about this, so actually I don't know whether it is a regression or not.

[gaborbernat]

This was left undefined on purpose and we did the change to have shorter commands. Is this a problem for you? Why?

[stanislavlevin]

This breaks some assumptions.
For example.

1. https://github.com/freeipa/freeipa

There is a custom installation command.

install_command={toxinidir}/.tox-install.sh wheel_bundle {envpython} {envsitepackagesdir} {packages}

which is used to build wheels. Now make fails due to missing path (relative).
Of course, this might be fixed using realpath.

2. being a maintainer of tox and many other Python packages in ALTLinux I like to test ones during RPM build using something like this:

sed -i '/\[testenv\]$/a whitelist_externals =\                                  
    \/bin\/cp\                                                                  
    \/bin\/sed\                                                                 
commands_pre =\                                                                 
    cp %_bindir\/py.test3 \{envbindir\}\/pytest\                                
    sed -i \x271c #!\{envpython\}\x27 \{envbindir\}\/pytest' tox.ini

export PIP_NO_INDEX=YES                                                         
export TOXENV=py%{python_version_nodots python},py%{python_version_nodots python3}
tox.py3 --sitepackages -p auto -o -rv 

This is exposed as:

py37 run-test-pre: commands[0] | cp /usr/bin/py.test3 /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin/pytest
[24219] /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1$ /bin/cp /usr/bin/py.test3 /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin/pytest
py37 run-test-pre: commands[1] | sed -i '1c #!/usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin/python' /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin/pytest
[24220] /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1$ /bin/sed -i '1c #!/usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin/python' .tox/py37/bin/pytest

As you can see the very first {envbindir} is expanded as /usr/src/RPM/BUILD/python-module-pytest-cov-2.7.1/.tox/py37/bin while the last one as .tox/py37/bin.

This is not an error for now.
But it's should be predictable as well as the related documentation should be improved.
What do you think?

GitHub

freeipa/freeipa

Mirror of FreeIPA, an integrated security information management solution - freeipa/freeipa

[stanislavlevin]

Any news?

[gaborbernat]

Feel free to make a PR 👌with proposed solution

[stanislavlevin]

As you correctly marked it's a question.
So, should the path be relative or absolute? why?
Based on your opinion I could improve docs.
Thank's.

[gaborbernat]

I would say absolute all injected as harder to mess up with changing the current working directory.

[tucked]

I just hit this while dealing with pypa/pipenv#4588

pipenv chokes on relative paths, which was baffling because tox showed that it was passing in an absolute path!
Regardless of whether tox uses relative or absolute paths, I'd strongly advocate showing what it's actually doing 🙏

[tucked]

I'll note that I also cannot work around this with {env:PWD}/{envpython} because that causes {envpython} not to be relative 😩
This seems to work tho:

/{envpython}

[gaborbernat]

This now has been reverted in the absolute path in tox 4.