Git and Mercurial revision checkout not working under Windows

[2xB]

The revision checkout routines located in setuptools/setuptools/package_index.py, functions _download_git and _download_hg, rely on (cd %s && <command>) to execute <command> in %s. This does not work under Windows, as simply tested by executing cd .. && echo %cd%.

Further reading: https://stackoverflow.com/q/55641332

[benoit-pierre]

Yes, apparently Windows does not do delayed expansion by default, that does not mean that the second command is not executed in the right directory, just that variable expansion for all commands is done prior to execution (and not independently for each command as it's about to be executed). A simple test to check the behavior is using cd .. && dir: the listing will be for the parent directory.

In any case, if you have an issue with setuptools' VCS code, you should provide a reproducible example, but really it's best to use and rely on pip instead, as its VCS code is more complete (and better tested).

[2xB]

Okay, a reproducible example:

1. Take this setup.py: https://gist.github.com/2xB/d82f7e1534484dcbf151f1b57b73e484
2. Try python setup.py install, notice

Checking out 2xB-patches
fatal: Not a git repository (or any of the parent directories): .git"
[...]
error: The 'pyqtgraph==0.11.0.dev0+g7201bae' distribution was not found and is required by test-t

The suggested change fixes this behaviour.

PS: Thanks for the quick response! I thought about using pip, although I would also be happy to hereby improve setuptools' VCS code ^^

[2xB]

PPS: I should have mentioned that the example should of course be executed under Windows.
And yes, this seems to be a different issue than the output of cd .. && echo %cd%, as it is not issued by expansion as benoit-pierre explained...

[benoit-pierre]

Dependency links are no longer supported by pip, you should use a direct URL requirement instead (see PEP 540):

from setuptools import setup

requirements = [
    "pyqtgraph @ git+https://github.com/2xB/pyqtgraph@2xB-patches",
    ]

setup(
    # Metadata
    name='test_t',
    version='0.0.1',
    install_requires=requirements,
)

And then of course use pip to install, not python setup.py install.

As for the error you're getting with the original setup.py, I can't reproduce it, but get another one:

Doing git clone from https://github.com/2xB/pyqtgraph to C:\Users\IEUser\AppData\Local\Temp\easy_install-9fmt48c1\pyqtgraph@2xB-patches
Checking out 2xB-patches
error: pathspec '2xB-patches' did not match any file(s) known to git.

So the wrong version of pyqtgraph gets installed: pyqtgraph-0.11.0.dev0+ge5e103d.

[2xB]

Your setup.py file executed via pip install test_t gives me the following error:
Direct url requirement (like pyqtgraph@ git+https://github.com/2xB/pyqtgraph@2xB-patches) are not allowed for dependencies [EDIT: see below]

Regarding the error you get from my file: This is because in your tests the test setup.py you execute is inside an other git repository, in which the git checkout is then executed. To prove my point, you can check where the checkout of 2xB-patches is executed by simply changing the line 900 in setuptools/package_index.py to os.system("(cd %s && git rev-parse --show-toplevel)" % (
and commenting out line 903.

[2xB]

Correcting the first part of my last comment: Ough, I used an old version of pip, sorry. After updating, your setup.py worked seamlessly (using pip install . of course). One question though: Can one specify the version of the dependency in the new requirement format? With my old-style setup.py, I could specify a version tag to cause an update of the dependency once necessary. I fail to reproduce this with the URL requirement: If the dependency is installed over a git repository once, it will allways be recognized as up-to-date whatever I write in the setup.py. Does updating URL dependencies via setup.py only work with setuptools' dependency links?

[EDIT] I would btw. understand if this was a question I could better ask elsewhere, but it currently decides how important the setuptools dependency links are for me and how happy I would be in seeing the pull request or a similar fix merged.

[jaraco]

I hadn't read this thread before accepting the PR (I probably should have). I do think the proposed approach is more robust than the one that was used previously, so I'm happy to see it incorporated, even if it is using behavior in discouraged parts of the codebase.

I did not intend to countermand Benoit's opinion. Happy to revert at your discretion, @benoit-pierre.

[2xB]

Further information: I was not able to reproduce this behaviour in Windows 7, it seemingly is a Windows 10 thing.

Thank you again both of you for this quick reaction and also for this very interesting discussion! I will now also further look into how to incorporate @benoit-pierre s comments into my projects.

[benoit-pierre]

@jaraco: well, using git -C is cleaner, although the code still suffer from some of the same issues (no escaping of arguments for example). I don't think there's a point in spending more time investigating the issue when most of this code is deprecated by the use of pip.

@2xB: the unfortunate truth is direct URL requirements are a very poor replacement for dependency links. In fact, I think there's pretty much useless, see discussion here: pypa/pip#5898.

In your case, I would change the requirement to pyqtgraph==0.11.0.dev0+g7201bae, forget about dependency links or using a direct URL requirement, and then either maintain a custom index if your creating private packages or make the wheel / source distribution available at a public URL so you can ask users to install using pip install -f public_url_for_patched_pyqtgraph_version mypackage. There's no way to get pip to install the custom version with a simple pip install mypackage invocation.