Local installation from sources - failing due to version not being parsable

[zerothi]

I did the following things:

1. Download latest release tar from github here
2. Extracted tar file
3. Ran

python3 setup.py install --prefix=<>

4. Got this error:

Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    setup(use_scm_version=True)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools/dist.py", line 445, in __init__
    _Distribution.__init__(self, {
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/distutils/dist.py", line 292, in __init__
    self.finalize_options()
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools/dist.py", line 734, in finalize_options
    ep.load()(self, ep.name, value)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools_scm/integration.py", line 17, in version_keyword
    dist.metadata.version = _get_version(config)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools_scm/__init__.py", line 148, in _get_version
    parsed_version = _do_parse(config)
  File "/opt/gnu/9.3.0/python/3.8.3/lib/python3.8/site-packages/setuptools_scm/__init__.py", line 110, in _do_parse
    raise LookupError(
LookupError: setuptools-scm was unable to detect version for '/home/nicpa/installation/bash-build/.compile/py-3.8/xarray-0.16.0'.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj

I think this is because the tar's created by tagging does not contain the version information anywhere.
The setuptools_scm reads versions according to this

As far as I can see, one should create a PKG-INFO with this content:

Version: <version>

I then tried this and then it worked. It would be ideal if the release pages have a fully functional installation procedure. This is mainly useful for non-pip installations etc.

[max-sixty]

Thanks @zerothi , for both the case and the link.

Do you happen to know how to automatically add the version number to the PKG-INFO file? Or does it need to be done manually?

[zerothi]

Hmm, sadly I know of no other way than uploading a new release file, or have it manually added to the setup.py with setup(version=<>, ...).
Alternatively this could be automated with github-actions.

[keewis]

is there a reason why you can't use the sdist archive from PyPI? I think that archive contains the desired PKG-INFO file.

Edit: I'm not quite sure if we can make github run python setup.py sdist or python -m pep517.build --source . (once we added a pyproject.toml) to generate the tarball.

[zerothi]

@keewis you're absolutely correct. That should of course work. It is just really annoying when I have automated scripts where I could merely change the version. On pypi (pythonhosted) there is a hash in the download path which requires me to do manual labour ;)

For now it is rather simple for me since I have just inserted a paste into the PKG-INFO file locally, but I guess others who (erroneously) download the release from GitHub will experience the same...

[keewis]

pip parses the pages on https://pypi.org/simple (for xarray: https://pypi.org/simple/xarray); you could use a html parser to extract the exact location from there (but then you'd be partially reimplementing a packaging frontend like pip or poetry)

If you don't mind the additional step, you could also just clone the repository and generate the archive yourself:

git clone https://github.com/pydata/xarray
cd xarray
git checkout v0.16.0
python setup.py sdist  # or python setup.py install --prefix=<>

[zerothi]

Thanks for the simple path, I didn't know this.

However, this is trivially solved for me by adding echo "Version: $version" > PKG-INFO to my automated build-script.
So this is more for others to know.
But perhaps it should be clear that release tar's on github are not installable.

[keewis]

I guess we could try to add a github workflow that builds the tarball using python setup.py or pep517.build. I'll experiment with create-release and upload-release-asset, but no promises.

[zerothi]

@keewis for future reference I have been digging into the pep517 business a little more. Please see here sigma-py/orthopy#97

My take home is that one should generally try to avoid the direct use of pep517 and let pip handle it.
I.e. pip install --prefix<> .
However, this (i think) still requires the version to be present in one way or the other.

Thanks!

[keewis]

You may want to keep an eye on separate pep517 front-ends, like python-build (for building sdist and wheels, replacing pep517.build) and installer (for installing sdist archives and wheels). Both are experimental and will be moved to the PyPA organization once they're stable enough (and probably reused by pip).

However, this (i think) still requires the version to be present in one way or the other.

Yep, neither pip nor installer get the version right when installing from the zip generated by github (which is just a snapshot of the repository, without the .git directory), so we'd still have to make github call the sdist builder (pep517.build or python setup.py sdist, for now).

[zerothi]

Exactly, and thanks for the pointers!

[keewis]

I guess we can close this; setuptools_scm doesn't work with archives built using git archive.