-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] pip install -e
broken since setuptools==60.0.0
#3301
Comments
I am facing the same issue. Python version is 3.10 and I am running Arch Linux. I can't downgrade to lower versions of setuptools since I am facing different issues with them - the package name comes as UNKNOWN (see this issue). Currently I am managing by manually adding the Python source directory to path. Would really appreciate a fix for this issue. Thanks! |
Hi guys, thank you for reporting this. Could you try again with the latest version of pip/setuptools? cd /tmp
git clone https://github.com/ivsanro1/gft-ner
cd gft-ner
virtualenv -p py37 .venv
.venv/bin/python -m pip install -U pip setuptools
.venv/bin/python -m pip install -e .
cd ..
gft-ner/.venv/bin/python -c 'import ner; print(ner.__version__)' # => 1.0.0 |
I also can run this without a virtual environment (in a container): > docker run --rm -it python:3.7.13-alpine ash
apk add --update git
python -m pip install -U pip setuptools
cd /tmp
git clone https://github.com/ivsanro1/gft-ner
cd gft-ner
pip install -e .
cd ..
python -c 'import ner; print(ner.__version__)' # => 1.0.0 Here I am purposefully avoiding using the global |
The latest version of setuptools |
It still fails in my Ubuntu 18.04 environment. Maybe it's the way in which I'm installing Python3? Here's my FROM ubuntu:18.04
RUN apt-get update
RUN apt install -y git
################## INSTALL PYTHON & PIP ##################
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:deadsnakes/ppa
RUN apt-get install -y python3.7 python3.7-dev
# If run before install pip, so pip installs for 3.7
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 1
RUN apt-get install -y python3-pip
RUN ln -s /usr/bin/pip3 /usr/bin/pip
##########################################################
RUN python3 -c 'import sys; print(f"\n\n\nPYTHON VERSION: {sys.version}\n\n\n")'
# From here it's basically @abravalheri's steps adapted to the environment
RUN python3 -m pip install -U pip setuptools
WORKDIR /tmp
RUN git clone https://github.com/ivsanro1/gft-ner
WORKDIR /tmp/gft-ner
RUN pip install -e .
WORKDIR /
RUN echo `python3 -c 'import ner; print(f"\n\n\nLIBRARY IS INSTALLED - VERSION: {ner.__version__}\n\n\n")'` Output (only error):
But still, even if my
to:
The output with
|
@ivsanro1, that is a possibility... I manage to get things working on the selected version of Ubuntu, with the following commands: > docker run --rm -it ubuntu:18.04 bash
apt-get update
apt-get install -y software-properties-common
add-apt-repository -y ppa:deadsnakes/ppa
apt-get update
apt-get install -y git python3.7 python3.7-dev
apt-get install -y python3.7-venv
# python3.7-venv will provide the 'ensurepip' package as a side effect,
# which in turn is used to install pip
python3.7 -m ensurepip
# In Debian-based systems, there is a well-know issue with pip
# which makes old-versions of setuptools to leak into the build
# environment (see pypa/pip#6264), so let's uninstall and reinstall
# setuptools just in case...
python3.7 -m pip uninstall -y setuptools
python3.7 -m pip install -U setuptools
git clone https://github.com/ivsanro1/gft-ner /tmp/gft-ner
cd /tmp/gft-ner
python3.7 -m pip install -e .
python3.7 -c 'import ner; print(ner.__version__)' The output is python3.7 -m pip list
Package Version
------------------- ---------------
pip 22.0.4
PyGObject 3.26.1
python-apt 1.6.5+ubuntu0.7
setuptools 62.3.2
unattended-upgrades 0.1 |
Thank you @abravalheri . This way it works. Thank you for your time |
@ivsanro1 the Debian patches for Python (also present in Ubuntu and deadsnakes) are very complicated and they heavily customize the installation directories for setuptools... That is why I demonstrated first with a virtual environment and then with Alpine. I am not 100% sure that I am not forgetting anything in the example above, but just to be on the safe side I always recommend using a virtual environment, even if you are using containers (there is some debate in the community if that is really necessary, but I believe that everyone agrees that using a virtual environment inside a container does not hurt). |
@abravalheri I think you're right and the best is to use a virtual env even inside of a docker container. I will be using a virtual environment for now By the way, I think I rushed yesterday to close the issue, as in your example, it works because you're importing the library when your working directory has the After I found that the problem probably relies in the way I'm installing python and pip, installing from deadsnakes repo is quite common to find in Google (in fact, is the only way I've read), so I thought there would be more people in the future that could run to this issue. For now my problem is solved because I will be using virtual environments, but in that environment made of |
Thank you very much @ivsanro1 for re-checking that. I forgot to jump outside of the directory in the last example 😝 . Between v60 and v61 there have been a series of changes imported from I tried these steps: > docker run --rm -it ubuntu:18.04 bash
apt-get update
apt-get install -y git wget python3.7 python3.7-dev python3.7-distutils
# I am using the official Debian distribution of python3.7
# to reduce the "surface" for the bug (minimise the moving parts).
# The deadsnakes distribution should not be very different...
wget https://bootstrap.pypa.io/get-pip.py -P /tmp
python3.7 /tmp/get-pip.py
# In Debian-based systems, there is a well-know issue with pip
# which makes old-versions of setuptools to leak into the build
# environment (see pypa/pip#6264), so let's uninstall and reinstall
# setuptools just in case...
python3.7 -m pip uninstall -y setuptools
python3.7 -m pip install -U pip setuptools
git clone https://github.com/ivsanro1/gft-ner /tmp/gft-ner
cd /tmp/gft-ner
python3.7 -m pip install -e . Using the following commands we can see that the ls /usr/lib/python3.7/site-packages/
# easy-install.pth ner.egg-link # <-- installed in this folder
python3.7 -m site
# sys.path = [
# '/tmp',
# '/usr/lib/python37.zip',
# '/usr/lib/python3.7',
# '/usr/lib/python3.7/lib-dynload',
# '/usr/local/lib/python3.7/dist-packages', # <- `site-packages` is missing
# '/usr/lib/python3/dist-packages',
# ]
# USER_BASE: '/root/.local' (doesn't exist)
# USER_SITE: '/root/.local/lib/python3.7/site-packages' (doesn't exist)
# ENABLE_USER_SITE: True What seems to be happening here is the following:
|
I don't think there is much we can do from the The Debian maintainers seem to be aware that Meanwhile, users can do the following to workaround of this issue: SETUPTOOLS_USE_DISTUTILS=stdlib python3.7 -m pip install -e . |
I believe I have the same issue. I'm on a Mac.
With setuptools With the latest setuptools ( Adding |
Hi @thomie, this issue is Debian specific. Maybe the issue you are facing is related to the existence of a |
Hello @abravalheri, Thank you for your response. I don't think my problem is related to (the not reading of) The problem I'm facing is that the egg-link gets installed into "{userbase}/lb/python{py_version_short}/site-packages", instead of "{userbase}/lb/python/site-packages". I can reproduce the problem with a fresh So I think the problem has been fixed, and it was most likely not an issue with setuptools, but rather something to do with sysconfig.py or the brew [email protected] formula. |
Thank you very much @thomie for the update.
Unfortunately it is a bit complicated to trace back and isolate what actually is causing the problem... I believe that the maintainers for the Given the following observation:
I am thinking that this can actually be related to |
The issue appears when doing 'pip install -e <package>', the package does not show up when doing 'pip freeze'. This is caused by Ubuntu installing a patched version of distutils (see pypa/distutils/issues/17 and pypa/setuptools/issues/3301; see also pypa/setuptools/issues/3625 for when this will stop working). This adds a workaround. Better would be to create a virtualenv and install into that and be insulated from odd OS upstream choices.
It is now April of 2023 and I am now running into this issue. I am using the workaround of creating a venv inside my container which is undesirable in my case. In my case I've tracked that my package with the new setup tools was added to:
Where the old one added to:
But sys.path does not load the paths from /usr/lib so it gets lost. So setuptools does not act in parallel with whatever is starting python. Is there any timeline on when this might be resolved? |
Hi @sei-amellinger, have you considered the information available in #3301 (comment) and in #3301 (comment) ?
If I understood what you are describing correctly, there seems to be no action point on the setuptools side. |
Thanks for the quick response! When parsing this bug report I saw the issue 3625 issue as deprecating the used of "SETUPTOOLS_USE_DISTUTILS=stdlib" and interpreted it as an approach to try. So, didn't try it. So on your suggestion I went back and tried it (#3301) and it DOES work for me. (Woot!) How long is this going to be a viable workaround if the plan is to deprecate that switch? And advice? Thanks! |
Hi @sei-amellinger, setuptools is driven by volunteer efforts which means that we are limited in terms of commitments. Realistically speaking, we are going to support We do however, bump the "major" version in a release, every time we know there is a breaking change or a "potential" breaking change. This allows users to "freeze" their setup if they want to be absolutely sure about backward compatibility. If you use [build-system]
requires = ["setuptools==67.6.1"] # or less dramatically "setuptools~=67.6"
build-backend = "setuptools.build_meta" Please consider however that a virtual environment is undoubtedly the safest approach. |
That's not always feasible, at the least, i see those use cases:
In both cases, debian specific patches will interfere, and that would be really a wrong move to ignore such a widespread OS instead of making things just retocompatible...
|
I seem to have a similar problem under MacOS. It seems to only create the |
setuptools version
Since setuptools==60.0.0
Python version
3.7.13
OS
Ubuntu 18.04
Additional environment information
No response
Description
I have a python library in local, which I'm developing. Therefore, I'm interested in installing it with
pip install -e
as I've always done, to develop it while it's installed.However, today I updated
setuptools
from46.1.3
to the newest (62.1.0
) and I realized that even ifpip install -e /path/to/my/lib
reports no error in output, but I cannot import the library in python.I've tracked down the version that broke this and it looks like it's
60.0.0
.Install
setuptools==59.8.0
:Install library with
pip install -e
:Try to import library in python:
It works with
setuptools==59.8.0
.Now let's do the same with
setuptools==60.0.0
:Install library with
pip install -e
:Try to import library in python:
Expected behavior
The installed library via
pip install -e
should be able to be imported like withsetuptools < 60
How to Reproduce
pip install setuptools==60.0.0
git clone https://github.com/ivsanro1/gft-ner
pip install -e
. e.g.pip install -e ~/repos/gft-ner
python3 -c "import ner"
Output
The text was updated successfully, but these errors were encountered: