inconsistent uninstall/install behavior

[rgommers]

• Pip version: 9.0.1
• Python version: 2.7
• Operating system: Linux

Description:

I'm trying to install numpy from a local git repo, after first manually uninstalling it to work around another pip issue. It looks like the metadata for installed packages that pip keeps is used in a way that allows inconsistencies - it tells me both that numpy is not installed and that it is installed.

What I've run:

pip uninstall numpy
...
Proceed (y/n)? y
    Successfully uninstalled numpy-1.13.0

$ pip uninstall numpy
Cannot uninstall requirement numpy, not installed

$ pip install . --user
Processing /home/rgommers/Code/numpy
  Requirement already satisfied (use --upgrade to upgrade): numpy==1.13.0 from file:///home/rgommers/Code/numpy in /home/rgommers/Code/numpy

Forcing an upgrade succeeds, but the above is still a real issue.

$ pip install . --user -U
Processing /home/rgommers/Code/numpy
Installing collected packages: numpy
  Running setup.py install for numpy ... done
Successfully installed numpy-1.13.0

[pradyunsg]

Thanks for filing this. #1056 might be related to this.

to work around another pip issue.

Just curious, what's the other pip issue you're referring to here? :)

[rgommers]

Thanks for filing this. #1056 might be related to this.

I don't think it's related. I always install with --user only, if I forget it installs will fail and I certainly never install with sudo.

Just curious, what's the other pip issue you're referring to here? :)

IIRC that pip install . --user doesn't uninstall the installed version of the project and installs . if the version number of . is not higher than that of the installed version.

[pradyunsg]

I don't think it's related.

I mixed this with another issue. Sorry.

doesn't uninstall the installed version of the project and installs . if the version number of . is not higher than that of the installed version.

Yep. That's #536. (just adding a cross-link)

---

I'll try to reproduce this (with a lighter package than numpy).

[pradyunsg]

I couldn't reproduce this. I think I'm missing something. :/

Could you provide steps to reproduce this issue?

---

PS: I thought I should mention this to you; pip install --no-deps --force-reinstall . should work for re-installing numpy (it's a workaround for #536).

[rgommers]

Could you provide steps to reproduce this issue?

That's hard, because it depends on the history of what I did before I executed the first command in pip uninstall numpy (what I don't know). But I've seen it multiple times.

The best thing to do is look at how pip actually does these lookups for installed packages, because it'll be a pretty obviously incorrect implementation. It should call the exact same function/method on the same object, which will make it impossible to get not installed and Requirement already satisfied back-to-back for the same package.

If you tell me where pip keeps its installed packages metadata, I can dig into that.

[rgommers]

thought I should mention this to you; pip install --no-deps --force-reinstall . should work for re-installing numpy

thanks:)

[pfmoore]

If you tell me where pip keeps its installed packages metadata, I can dig into that.

It looks at dist-info/egg-info directories on sys.path. This could be one of those cases where you somehow ended up with two? Maybe an egg-info from a setup.py develop install and a dist-info from pip? The dist-info doesn't record the egg-info as it wasn't installed by pip, so uninstall doesn't remove it, and now pip sees an egg-info which says numpy is present when it actually isn't?

[rgommers]

Thanks @pfmoore. Can't find any egg-info directories on my path for numpy, and I never use setup.py develop. But can't reproduce the issue anymore, so hard to say for sure that this wasn't it.

The behavior you describe seems wrong though - it really should be symmetric. Or at the very least, the message Cannot uninstall requirement numpy, not installed should be fixed, it's very misleading. If egg-info is detected, it should say requirement numpy appears to be installed by a program other than pip, so cannot uninstall. And ideally with a .egg-info location, so a user can go and remove it manually.

[pradyunsg]

FWIW, I just tried out pip install ., python setup.py install and python setup.py develop for a basic package foo;

pip uninstall foo detected and uninstalled them one by one over 3 runs.

[rgommers]

Did some more testing with current pip master:

$ ls   # in site-packages
scikit_learn-0.16.1-py2.7.egg-info
scikit_learn-0.17.dev0.egg-info
scikit_learn-0.18.dev0.egg-info
scikit_learn-0.19.dev0.dist-info
..
sklearn/

$ pip uninstall scikit-learn
Uninstalling scikit-learn-0.19.dev0:
  Would remove:
    /home/rgommers/.local/lib/python2.7/site-packages/scikit_learn-0.19.dev0.dist-info/*
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/*
  Would not remove (might be manually added):
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/_build_utils.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/_build_utils/cythonize.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/externals/odict.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/metrics/metrics.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/preprocessing/_weights.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/tests/test_lda.py
    /home/rgommers/.local/lib/python2.7/site-packages/sklearn/tests/test_qda.py
Proceed (y/n)? y
  Successfully uninstalled scikit-learn-0.19.dev0

$ pip uninstall scikit-learn
Cannot uninstall 'scikit-learn'. It is a distutils installed project and thus
we cannot accurately determine which files belong to it which would lead to
only a partial uninstall.

$ pip install scikit-learn --user
Requirement already satisfied: scikit-learn in /home/rgommers/.local/lib/python2.7/site-packages (0.18.dev0)

So for left over .egg-info the message is fairly informative. The message Cannot uninstall requirement <>, not installed in my initial report is not present anymore. Therefore I'll close this. If I see it again, will investigate in more detail.

[elmerehbi]

@rgommers I am facing this problem too. I am using Python 2.7 and have scikit-learn 0.17. I faced an import error when running from sklearn.model_selection import train_test_split.

I searched for this and a solution is to update sklearn to 0.18. I tried this but I got the error corresponding to this reported bug. How can I solve this?

[elmerehbi]

Solved with pip install --upgrade --force-reinstall --user scikit-learn. I simply added the --user flag.