[BUG] Can't Install from Cached source and wheel in Container

[madhavajay]

Describe the bug
I am trying to install Bottleneck as a dependency in a container using a manually created pip cache.

To Reproduce
To assist in reproducing the bug, please include the following:

1. Command/code being executed

$ cd /tmp
$ python3 -m pip download Bottleneck -d ./ -v
$ ls
Bottleneck-1.3.2.tar.gz numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl
$ python3 -m pip install Bottleneck --find-links /tmp --no-index

2. Python version and OS

Docker Container FROM nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04
Linux ab183940868d 4.19.76-linuxkit #1 SMP Thu Oct 17 19:31:58 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Python 3.6.9 (default, Nov  7 2019, 10:44:02)
[GCC 8.3.0] on linux

3. pip version

pip 20.0.2 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

4. Output of pip list or conda list

Package       Version
------------- -------
asn1crypto    0.24.0
cryptography  2.1.4
idna          2.6
keyring       10.6.0
keyrings.alt  3.0
numpy         1.16.0
pip           20.0.2
pycrypto      2.6.1
pygobject     3.26.1
pyxdg         0.25
SecretStorage 2.3.1
setuptools    39.0.1
six           1.11.0
wheel         0.30.0

Expected behavior
A clear and concise description of what you expected to happen.
Package should install.

Additional context
Error output:

Looking in links: /tmp
Processing ./Bottleneck-1.3.2.tar.gz
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-1z33ubip/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'numpy==1.13.3; python_version=='"'"'2.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.5'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'2.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.5'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"''
       cwd: None
  Complete output (12 lines):
  Ignoring numpy: markers 'python_version == "2.7" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.5" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "2.7" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.5" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.6" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system == "AIX"' don't match your environment
  Looking in links: /tmp
  ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-1z33ubip/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'numpy==1.13.3; python_version=='"'"'2.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.5'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'2.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.5'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"'' Check the logs for full command output.

[qwhelan]

I'd recommend trying upgrading setuptools - your version is about two years old

[madhavajay]

$ pip install -U pip setuptools

Requirement already up-to-date: pip in /usr/local/lib/python3.6/dist-packages (20.0.2)
Collecting setuptools
Using cached setuptools-46.0.0-py3-none-any.whl (582 kB)
Installing collected packages: setuptools
Attempting uninstall: setuptools
Found existing installation: setuptools 39.0.1
Uninstalling setuptools-39.0.1:
Successfully uninstalled setuptools-39.0.1
Successfully installed setuptools-46.0.0

$ pip list

Package Version

---

asn1crypto 0.24.0
cryptography 2.1.4
idna 2.6
keyring 10.6.0
keyrings.alt 3.0
pip 20.0.2
pycrypto 2.6.1
pygobject 3.26.1
pyxdg 0.25
SecretStorage 2.3.1
setuptools 46.0.0
six 1.11.0
wheel 0.30.0

Same again:

root@3b2e48aa07ff:/tmp# python3 -m pip install Bottleneck --find-links /tmp --no-index

Looking in links: /tmp
Processing ./Bottleneck-1.3.2.tar.gz
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ergljuop/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'numpy==1.13.3; python_version=='"'"'2.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.5'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'2.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.5'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"''
       cwd: None
  Complete output (12 lines):
  Ignoring numpy: markers 'python_version == "2.7" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.5" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system != "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "2.7" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.5" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.6" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version == "3.7" and platform_system == "AIX"' don't match your environment
  Ignoring numpy: markers 'python_version >= "3.8" and platform_system == "AIX"' don't match your environment
  Looking in links: /tmp
  ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ergljuop/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'numpy==1.13.3; python_version=='"'"'2.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.5'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.13.3; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'2.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.5'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"'' Check the logs for full command output.

[qwhelan]

I was able to reproduce the error with a local Dockerfile and it seems that the underlying issue is that pip download does not respect PEP 517 declared build dependencies.

Adding them in fixes the issue. Dockerfile:

FROM nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04
RUN apt-get update
RUN apt-get install -y python3 python3-pip
RUN pip3 install -U pip
RUN cd /tmp && python3 -m pip download Bottleneck setuptools numpy==1.13.3 wheel -d ./
RUN cd /tmp && python3 -m pip download numpy==1.16.0 -d ./
RUN ls -alh /tmp
RUN python3 -m pip install Bottleneck numpy==1.16.0 --find-links /tmp --no-index
RUN pip3 list

Truncated output:

Step 8/9 : RUN python3 -m pip install Bottleneck numpy==1.16.0 --find-links /tmp --no-index
 ---> Running in 2807bda2a3a9
Looking in links: /tmp
Processing /tmp/Bottleneck-1.3.2.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Processing /tmp/numpy-1.16.0-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: Bottleneck
  Building wheel for Bottleneck (PEP 517): started
  Building wheel for Bottleneck (PEP 517): finished with status 'done'
  Created wheel for Bottleneck: filename=Bottleneck-1.3.2-cp36-cp36m-linux_x86_64.whl size=330986 sha256=b6feca6de5d37b2e9250d36d141bca39b9aa647cab5c26fcab5a319c5200b94a
  Stored in directory: /root/.cache/pip/wheels/f7/80/d0/65d4c1c1bb9ace57263c5cdf6b496aee38bc7bb8757b882fcd
Successfully built Bottleneck
Installing collected packages: numpy, Bottleneck
Successfully installed Bottleneck-1.3.2 numpy-1.16.0
Removing intermediate container 2807bda2a3a9
 ---> 5573ccbc0b8b
Step 9/9 : RUN pip3 list
 ---> Running in 4e654e5cb4b3
Package       Version
------------- -------
asn1crypto    0.24.0 
Bottleneck    1.3.2  
cryptography  2.1.4  
idna          2.6    
keyring       10.6.0 
keyrings.alt  3.0    
numpy         1.16.0 
pip           20.0.2 
pycrypto      2.6.1  
pygobject     3.26.1 
pyxdg         0.25   
SecretStorage 2.3.1  
setuptools    39.0.1 
six           1.11.0 
wheel         0.30.0 

A slightly modified version of the above script shows we can reproduce the same error message with numpy, if we force a PEP 517 build via --no-binary. Otherwise, this issue is sidestepped via .whl download:

FROM nvidia/cuda:10.0-cudnn7-runtime-ubuntu18.04
RUN apt-get update
RUN apt-get install -y python3 python3-pip
RUN pip3 install -U pip
RUN cd /tmp && python3 -m pip download numpy --no-binary numpy -d ./
RUN ls -alh /tmp
RUN python3 -m pip install --no-binary numpy numpy --find-links /tmp --no-index
RUN pip3 list

And error:

Step 7/8 : RUN python3 -m pip install --no-binary numpy numpy --find-links /tmp --no-index
 ---> Running in b34d3e92b764
Looking in links: /tmp
Processing /tmp/numpy-1.18.1.zip
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-6p0zdywu/overlay --no-warn-script-location --no-binary numpy --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'Cython>=0.29.14'
       cwd: None
  Complete output (3 lines):
  Looking in links: /tmp
  ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.6/dist-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-6p0zdywu/overlay --no-warn-script-location --no-binary numpy --only-binary :none: --no-index --find-links /tmp -- setuptools wheel 'Cython>=0.29.14' Check the logs for full command output.
The command '/bin/sh -c python3 -m pip install --no-binary numpy numpy --find-links /tmp --no-index' returned a non-zero code: 1

Closing as this seems to be a pip issue, not a bottleneck issue. I'll be opening an issue with them shortly.

Thanks again for reporting!

[erfannariman]

Was there a solution for this? Still getting the same error when installing bottleneck