GPU installation failure with pip

[jsoghigian]

Trying to install DeepConsensus on an HPC. I've got a conda environment created and an updated pip. On a GPU node...

Using the command: pip install deepconsensus[gpu]==1.2.0

Collecting deepconsensus[gpu]==1.2.0
  Using cached deepconsensus-1.2.0-py3-none-any.whl (164 kB)
Collecting pandas==1.5.1 (from deepconsensus[gpu]==1.2.0)
  Using cached pandas-1.5.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.0 MB)
Collecting tf-models-official==2.9.1 (from deepconsensus[gpu]==1.2.0)
  Using cached tf_models_official-2.9.1-py2.py3-none-any.whl (2.1 MB)
Collecting pyyaml==5.4.1 (from deepconsensus[gpu]==1.2.0)
  Using cached PyYAML-5.4.1.tar.gz (175 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      /tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!

              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.

              By 2023-Oct-30, you need to update your project and remove deprecated calls
              or your builds will no longer be supported.

              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************

      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running egg_info
      writing lib3/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib3/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/work/soghigian_lab/apps/conda/envs/deepconsensus/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/work/soghigian_lab/apps/conda/envs/deepconsensus/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/work/soghigian_lab/apps/conda/envs/deepconsensus/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 271, in <module>
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 1233, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 319, in run
          self.find_sources()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 327, in find_sources
          mm.run()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 549, in run
          self.add_defaults()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/command/egg_info.py", line 587, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/command/sdist.py", line 113, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 201, in get_source_files
        File "/tmp/pip-build-env-y6jm3vt2/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I can't (easily) use Docker... Any suggestions?

[danielecook]

Hi @jsoghigian I am currently trying to resolve this. Are you able to easily switch python versions? I think downgrading to Python 3.9 might resolve the issue, but I am still testing this.

[pgrosu]

Hi @jsoghigian,

Didn't see Daniel's comment, as I was typing -- so try that first before mine. This will be tricky to fix, but try the following afterwards:

1. Create a file called constraints.txt, and add the following to it:

cython < 3.0

2. The re-launch the requirements like this:

PIP_CONSTRAINT=constraints.txt pip3 install -r requirements.txt --no-cache

Let me know if this fixes it.

Thx,
~p

[jsoghigian]

thanks @danielecook , that allowed me to install deepconsensus and the test data ran flawlessly!