Env recreation during tox --devenv is broken for me
#1501
Labels
bug:normal
affects many people or has quite an impact
Comments
webknjaz
changed the title
tox --devenvtox --devenv is broken for me
|
I'm curious if it's somehow related to #1484 |
|
@asottile as the creator of this feature any idea why this happens? |
|
as far as I understand this is the same as the (open) problem with running tox recreate while having a tox env activated: tox -e py37 --notest
. .tox/py37/bin/activate
tox -e py37 --notest -r$ tox -e py37 --notest
GLOB sdist-make: /home/asottile/workspace/pre-commit/setup.py
py37 create: /home/asottile/workspace/pre-commit/.tox/py37
py37 installdeps: -rrequirements-dev.txt
py37 inst: /home/asottile/workspace/pre-commit/.tox/.tmp/package/1/pre_commit-1.21.0.zip
py37 installed: aspy.yaml==1.3.0,attrs==19.3.0,cfgv==2.0.1,coverage==5.0.3,identify==1.4.10,importlib-metadata==1.4.0,more-itertools==8.1.0,nodeenv==1.3.4,packaging==20.0,pluggy==0.13.1,pre-commit==1.21.0,py==1.8.1,pyparsing==2.4.6,pytest==5.3.3,pytest-env==0.6.2,PyYAML==5.3,six==1.14.0,toml==0.10.0,virtualenv==16.7.9,wcwidth==0.1.8,zipp==1.0.0
___________________________________ summary ____________________________________
py37: skipped tests
congratulations :)
$ . .tox/py37/bin/activate
$ tox -e py37 --notest -r
GLOB sdist-make: /home/asottile/workspace/pre-commit/setup.py
py37 recreate: /home/asottile/workspace/pre-commit/.tox/py37
ERROR: invocation failed (exit code 3), logfile: /home/asottile/workspace/pre-commit/.tox/py37/log/py37-0.log
================================== log start ===================================
The path /home/asottile/workspace/pre-commit/.tox/py37/bin/python3.7 (from --python=/home/asottile/workspace/pre-commit/.tox/py37/bin/python3.7) does not exist
=================================== log end ====================================
ERROR: InvocationError for command /home/asottile/opt/venv/bin/python3 -m virtualenv --no-download --python /home/asottile/workspace/pre-commit/.tox/py37/bin/python3.7 py37 (exited with code 3)
___________________________________ summary ____________________________________
ERROR: py37: InvocationError for command /home/asottile/opt/venv/bin/python3 -m virtualenv --no-download --python /home/asottile/workspace/pre-commit/.tox/py37/bin/python3.7 py37 (exited with code 3) |
|
Interesting... I use |
|
yeah tox probably needs to re-find the executable in the case of recreation (or do that more lazily / in a different order) -- I wonder if the rewrite covers this case |
|
I think (90% sure) the virtualenv rewrite will fix this if the issue is what @asottile says it is. |
|
virtualenv 20 is now released so this should be fixed. |
|
I'll check and report back here. |
|
No, it's still broken. Will post the logs shortly. |
$ python3 -m tox -vvvv --devenv venv-dev -e .dev-env
using tox.ini: ~/src/github/ansible/ansible-lint/tox.ini (pid 4966)
removing ~/src/github/ansible/ansible-lint/.tox/log
could not satisfy requires MissingDependency(<Requirement('setuptools>=41.4.0')>)
could not satisfy requires MissingDependency(<Requirement('pip>=19.3.0')>)
~/.pyenv/versions/3.8.2/bin/python3 (~/.pyenv/versions/3.8.2/bin/python3) is {'executable': '~/.pyenv/versions/3.8.2/bin/python3', 'name': 'python', 'version_info': [3, 8, 2, 'final', 0], 'version': '3.8.2 (default, Apr 16 2020, 16:41:34) \n[Clang 7.3.0 (clang-703.0.31)]', 'is_64': True, 'sysplatform': 'darwin'}
.tox uses ~/.pyenv/versions/3.8.2/bin/python3
using tox-3.14.6 from ~/.pyenv/versions/3.8.2/lib/python3.8/site-packages/tox/__init__.py (pid 4966)
.tox start: getenv ~/src/github/ansible/ansible-lint/venv-dev
.tox cannot reuse: no previous config ~/src/github/ansible/ansible-lint/venv-dev/.tox-config1
.tox create: ~/src/github/ansible/ansible-lint/venv-dev
removing ~/src/github/ansible/ansible-lint/venv-dev
setting PATH=~/src/github/ansible/ansible-lint/venv-dev/bin:~/.pyenv/versions/3.8.2/bin:/usr/local/Cellar/pyenv/1.2.8_1/libexec:~/.pyenv/plugins/python-build/bin:~/.pyenv/plugins/pyenv-virtualenv/bin:~/.pyenv/plugins/pyenv-update/bin:~/.pyenv/plugins/pyenv-installer/bin:~/.pyenv/plugins/pyenv-doctor/bin:~/.nvm/versions/node/v9.8.0/bin:~/.pyenv/plugins/pyenv-virtualenv/shims:~/.pyenv/shims:/Library/Frameworks/Python.framework/Versions/3.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Applications/kitty.app/Contents/MacOS
[4987] ~/src/github/ansible/ansible-lint$ ~/.pyenv/versions/3.8.2/bin/python3 -m virtualenv --no-download --python ~/.pyenv/versions/3.8.2/bin/python3 venv-dev
created virtual environment CPython3.8.2.final.0-64 in 514ms
creator CPython3Posix(dest=~/src/github/ansible/ansible-lint/venv-dev, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=~/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
.tox installdeps: setuptools >= 41.4.0, pip >= 19.3.0, tox >= 3.5.3
setting PATH=~/src/github/ansible/ansible-lint/venv-dev/bin:~/.pyenv/versions/3.8.2/bin:/usr/local/Cellar/pyenv/1.2.8_1/libexec:~/.pyenv/plugins/python-build/bin:~/.pyenv/plugins/pyenv-virtualenv/bin:~/.pyenv/plugins/pyenv-update/bin:~/.pyenv/plugins/pyenv-installer/bin:~/.pyenv/plugins/pyenv-doctor/bin:~/.nvm/versions/node/v9.8.0/bin:~/.pyenv/plugins/pyenv-virtualenv/shims:~/.pyenv/shims:/Library/Frameworks/Python.framework/Versions/3.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Applications/kitty.app/Contents/MacOS
[4988] ~/src/github/ansible/ansible-lint$ ~/src/github/ansible/ansible-lint/venv-dev/bin/python -m pip install 'setuptools >= 41.4.0' 'pip >= 19.3.0' 'tox >= 3.5.3'
Requirement already satisfied: setuptools>=41.4.0 in ./venv-dev/lib/python3.8/site-packages (46.1.3)
Requirement already satisfied: pip>=19.3.0 in ./venv-dev/lib/python3.8/site-packages (20.0.2)
Collecting tox>=3.5.3
Using cached tox-3.14.6-py2.py3-none-any.whl (81 kB)
Collecting six<2,>=1.14.0
Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting filelock<4,>=3.0.0
Using cached filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting py<2,>=1.4.17
Using cached py-1.8.1-py2.py3-none-any.whl (83 kB)
Collecting packaging>=14
Using cached packaging-20.3-py2.py3-none-any.whl (37 kB)
Collecting pluggy<1,>=0.12.0
Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.0.18-py2.py3-none-any.whl (4.6 MB)
Collecting toml>=0.9.4
Using cached toml-0.10.0-py2.py3-none-any.whl (25 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting appdirs<2,>=1.4.3
Using cached appdirs-1.4.3-py2.py3-none-any.whl (12 kB)
Processing ~/Library/Caches/pip/wheels/eb/4e/d2/a903d4184fb49e4ac06474d65715b129aee13d69f7d227e78e/distlib-0.3.0-py3-none-any.whl
Installing collected packages: six, filelock, py, pyparsing, packaging, pluggy, appdirs, distlib, virtualenv, toml, tox
Successfully installed appdirs-1.4.3 distlib-0.3.0 filelock-3.0.12 packaging-20.3 pluggy-0.13.1 py-1.8.1 pyparsing-2.4.7 six-1.14.0 toml-0.10.0 tox-3.14.6 virtualenv-20.0.18
.tox finish: getenv ~/src/github/ansible/ansible-lint/venv-dev after 5.13 seconds
.tox start: finishvenv
write config to ~/src/github/ansible/ansible-lint/venv-dev/.tox-config1 as 'd77efa7113a73c55f12bc892eea3236cf7e894434a17d76168548532e60ca119 ~/.pyenv/versions/3.8.2/bin/python3\n3.14.6 0 1 0\n00000000000000000000000000000000 setuptools >= 41.4.0\n00000000000000000000000000000000 pip >= 19.3.0\n00000000000000000000000000000000 tox >= 3.5.3'
.tox finish: finishvenv after 0.02 seconds
.tox start: provision
[4997] ~/src/github/ansible/ansible-lint$ ~/src/github/ansible/ansible-lint/venv-dev/bin/python -m tox -vvvv --devenv venv-dev -e .dev-env
using tox.ini: ~/src/github/ansible/ansible-lint/tox.ini (pid 4997)
~/src/github/ansible/ansible-lint/venv-dev/bin/python (~/src/github/ansible/ansible-lint/venv-dev/bin/python) is {'executable': '~/src/github/ansible/ansible-lint/venv-dev/bin/python', 'name': 'python', 'version_info': [3, 8, 2, 'final', 0], 'version': '3.8.2 (default, Apr 16 2020, 16:41:34) \n[Clang 7.3.0 (clang-703.0.31)]', 'is_64': True, 'sysplatform': 'darwin'}
.package uses ~/src/github/ansible/ansible-lint/venv-dev/bin/python
py38-ansible29 uses ~/src/github/ansible/ansible-lint/venv-dev/bin/python
py38-ansibledevel uses ~/src/github/ansible/ansible-lint/venv-dev/bin/python
py38-ansible28 uses ~/src/github/ansible/ansible-lint/venv-dev/bin/python
.dev-env uses ~/src/github/ansible/ansible-lint/venv-dev/bin/python
python (~/.pyenv/versions/3.8.2/bin/python) is {'executable': '~/.pyenv/versions/3.8.2/bin/python', 'name': 'python', 'version_info': [3, 8, 2, 'final', 0], 'version': '3.8.2 (default, Apr 16 2020, 16:41:34) \n[Clang 7.3.0 (clang-703.0.31)]', 'is_64': True, 'sysplatform': 'darwin'}
py36-ansible29 uses None
py36-ansibledevel uses None
py36-ansible28 uses None
python3.5 (/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5) is {'is_64': True, 'version_info': [3, 5, 4, 'final', 0], 'version': '3.5.4 (v3.5.4:3f56838976, Aug 7 2017, 12:56:33) \n[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]', 'executable': '/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5', 'name': 'python', 'sysplatform': 'darwin'}
py35-ansible29 uses /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
py35-ansible28 uses /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
py35-ansibledevel uses /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
python3.7 (~/.pyenv/shims/python3.7) is {'executable': '/usr/local/bin/python3.7', 'name': 'python', 'version_info': [3, 7, 0, 'final', 0], 'version': '3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 20:42:06) \n[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]', 'is_64': True, 'sysplatform': 'darwin'}
py37-ansibledevel uses /usr/local/bin/python3.7
py37-ansible28 uses /usr/local/bin/python3.7
py37-ansible29 uses /usr/local/bin/python3.7
using tox-3.14.6 from ~/src/github/ansible/ansible-lint/venv-dev/lib/python3.8/site-packages/tox/__init__.py (pid 4997)
skipping sdist step
.dev-env start: getenv ~/src/github/ansible/ansible-lint/venv-dev
.dev-env cannot reuse: attr base_resolved_python_path '~/.pyenv/versions/3.8.2/bin/python3'!='~/src/github/ansible/ansible-lint/venv-dev/bin/python'
.dev-env recreate: ~/src/github/ansible/ansible-lint/venv-dev
removing ~/src/github/ansible/ansible-lint/venv-dev
setting PATH=~/src/github/ansible/ansible-lint/venv-dev/bin:~/.pyenv/versions/3.8.2/bin:/usr/local/Cellar/pyenv/1.2.8_1/libexec:~/.pyenv/plugins/python-build/bin:~/.pyenv/plugins/pyenv-virtualenv/bin:~/.pyenv/plugins/pyenv-update/bin:~/.pyenv/plugins/pyenv-installer/bin:~/.pyenv/plugins/pyenv-doctor/bin:~/.nvm/versions/node/v9.8.0/bin:~/.pyenv/plugins/pyenv-virtualenv/shims:~/.pyenv/shims:/Library/Frameworks/Python.framework/Versions/3.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/usr/local/share/dotnet:/Applications/kitty.app/Contents/MacOS
ERROR: invocation failed (exit code 2)
ERROR: InvocationError for command ~/src/github/ansible/ansible-lint/venv-dev/bin/python -m virtualenv --no-download --python ~/src/github/ansible/ansible-lint/venv-dev/bin/python venv-dev (exited with code 2)
.dev-env finish: getenv ~/src/github/ansible/ansible-lint/venv-dev after 0.18 seconds
_____________________________________________________ summary ______________________________________________________
ERROR: .dev-env: InvocationError for command ~/src/github/ansible/ansible-lint/venv-dev/bin/python -m virtualenv --no-download --python ~/src/github/ansible/ansible-lint/venv-dev/bin/python venv-dev (exited with code 2)
.tox finish: provision after 1.13 seconds
ansible-lint on master [!] via 🐍 3.8.2 took 7s
$ python3 -m tox --version
3.14.6 imported from ~/.pyenv/versions/3.8.2/lib/python3.8/site-packages/tox/__init__.py
ansible-lint on master [!] via 🐍 3.8.2
$ python3 -m virtualenv --version
virtualenv 20.0.18 from ~/.pyenv/versions/3.8.2/lib/python3.8/site-packages/virtualenv/__init__.pyLooks like it's the same: creating a venv, then deciding to recreate it using the interpreter inside itself that effectively wipes it off and breaks:
|
|
I believe this should be fixed after #2318, let me know if still persists the issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
So I'm trying to set up the use of
--devenv. I must notice that the project doesn't havesetup.pyand is supposed to rely on new enough setuptools to handle this but this detail doesn't seem to be related to the error I'm seeing in any way.I added
to this config: https://github.com/ansible/ansible-lint/blob/f29b70a1cfb868814eaf9ec2b5ec78690ba11fb4/tox.ini
(actually I tried a few option combos, including w/o most of those options)
And here's tox version (I believe I tried it with a bit older version too).
$ python3 -m tox -vvvv --devenv venv-dev -e .dev-envThe interesting part is
so it removes
~/src/github/ansible/ansible-lint/venv-devand than tried to execute interpreter from that removed folder:InvocationError for command ~/src/github/ansible/ansible-lint/venv-dev/bin/python ....P.S. The
.dev-env cannot reuse:message is different depending on how I changetox.inibut that's probably a separate issue. For example, sometimes it claims that interpreter hashes differ (base_resolved_python_sha256).The text was updated successfully, but these errors were encountered: