Install procedure tries to uninstall Cantera-python version installed by system pkg manager

[band-a-prend]

System information

• Cantera version: 2.6.0b1
• OS: Gentoo linux
• Python: 3.9, 3.10

Problem description

New implemented Cantera-python installation procedure uses pip install with --force-reinstall option that tries to uninstall the Cantera version that was early placed by system package manager into system python site-packages directory. This attempt results in an error as system package manager blocks this attempt.

E.g. Gentoo package manager (portage) initially tries to install all files to specified stage_dir and then place it to system replacing early installed files if necessary. Thus the attempt to uninstall Cantera-python 2.5.1 files (installed by Gentoo package manager) by pip is failed as Gentoo package manager blocks such try while installing Cantera-python 2.6.0b1 files into python site-packages under site_dir build directory.
Linux distributives that uses similar aproach (e.g. Debian, Ubuntu, Fedora etc) could be affected this issue too.

The quick workaround for distributive maintainers is to patch interfaces/cython/SConscript and/or interfaces/python_minimal/SConscript files replacing --force-reinstall substring with --ignore-installed.

Otherwise the conditional check is required within interfaces/cython/SConscript and interfaces/python_minimal/SConscript to apply --force-reinstall if stage_dir option isn't used or --ignore-installed for pip install if stage_dir option is specified for Cantera scons build env.

[bryanwweber]

Thanks for reporting this @band-a-prend! I'll look into it before 2.6.0b2.

[band-a-prend]

The initial description looks complicated even for me now. So I try to clarify.

1. I have installed Cantera 2.5.1 package from gentoo repository.
2. I tried to prepare Cantera 2.6.0b1 package for gentoo for testing before final 2.6.0 release.
3. On package manager installation phase of prepared package scons install is ran with specified stage_dir and libdirname options.
4. For installation of python Cantera implementation scons calls pip command.
5. On package manager installation phase the pip installs files into site-packages directory under sandboxed (isolated) stage_dir (it's OK) and then tries to uninstall Cantera 2.5.1 from system (and got access denied).
6. Gentoo package manager doesn't allow to directly change files in system by call of external tool like pip. It assumed that package manager merge files to system from stage_dir on it's own.

[speth]

I tried something similar on Ubuntu 22.04:

1. Install Cantera 2.6.0b1 from the PPA
2. run scons build stage_dir=stage_here prefix=/usr python_prefix=/usr
3. run scons install, which outputs (in part):

   Successfully built Cantera
   Installing collected packages: Cantera
     Attempting uninstall: Cantera
       Found existing installation: Cantera 2.6.0b1
       Not uninstalling cantera at /usr/lib/python3/dist-packages, outside environment /usr
       Can't uninstall 'Cantera'. No files were found to uninstall.
     Creating /home/speth/cantera/stage_here/usr/local/bin
     changing mode of /home/speth/cantera/stage_here/usr/local/bin/ck2cti to 775
     changing mode of /home/speth/cantera/stage_here/usr/local/bin/ck2yaml to 775
     changing mode of /home/speth/cantera/stage_here/usr/local/bin/cti2yaml to 775
     changing mode of /home/speth/cantera/stage_here/usr/local/bin/ctml2yaml to 775
     changing mode of /home/speth/cantera/stage_here/usr/local/bin/ctml_writer to 775
   Successfully installed Cantera-2.6.0b1
   

Which, while it succeeds, still seems a bit fishy.

[band-a-prend]

@speth

2. run scons build
3. run scons install

This commands are ran as user and then scons call of pip should check user's site-package directory instead of system/global, isn't it?
But from the log I see that pip reports about found existing system Cantera installation and then about Can't uninstall 'Cantera'. No files were found to uninstall. It's strange and looks like check installation and trying to uninstall refers diferent paths.

I don't know the details of how Debian/Ubuntu package manager works but maybe this issue will reproduced if initially install Cantera 2.5.1 from PPA and then Cantera 2.6.0b1 from PPA.

Is PPA for Cantera 2.6.0b1 is public available?

I'm not sure if it's reproduced while reinstalling of the Cantera 2.6.0b1 for gentoo so need to do additional tests of it with providing logs.

[band-a-prend]

I realized that this issue could be observed only then reinstalling 2.6.0b1 via source based type of package over early installed cantera version takes place. And doesn't affect the binary packages.

I attach the part of gentoo build log with install error while reinstalling 2.6.0b1 over already installed 2.6.0b1: build_and_reinstall_2.6.0_beta1.log

The same error occurs on trying gentoo package installation of cantera-2.6.0_beta1 over early installed cantera-2.5.1.

@mefuller
as I assume you prepare the Fedora package, isn't you?
Could you please check if current issue affectes Fedora package while reinstalling cantera 2.6.0b1 being built from sources, i.e. on installation from rpm-src?

If it's only gentoo specific issue there is no problem patching for gentoo in place without any upstream changes.

[mefuller]

Just acknowledging that I have read this and will work on the issue:
I have a build regression (#1233) that needs to be sorted out before I can evaluate this issue

[speth]

This annoying behavior by pip is discussed in pypa/pip#4337, but it doesn't seem like behavior that's going to get resolved anytime soon.

I think the idea of adding --ignore-installed if we're either installing to the user site-packages directory or stage_dir is set would work at least in some cases. The case that's more complicated, though, is figuring out if python_prefix is pointing to a location that's either an existing Cantera installation (in which case, we should let pip install the old version) or somewhere else (in which case, it should not). I guess this can be inspected with pip show cantera, which will show something like:

Name: Cantera
Version: 2.6.0b1
Summary: Cantera is an open-source suite of tools for problems involving chemical kinetics, thermodynamics, and transport processes.
Home-page: https://cantera.org
Author: Cantera Developers
Author-email: [email protected]
License: BSD 3-Clause License
Location: /home/speth/.local/lib/python3.10/site-packages
Requires: ruamel.yaml, numpy
Required-by:

[band-a-prend]

Great thanks!
I confirm that the implemented check fixes the issue for packaging in my case.

[band-a-prend]

As usual I forgot to retest python_package = 'minimal' installation over early installed Cantera-minimal package. :(
And the similar changes implemented in #1239 are required for interfaces/python_minimal/SConscript with checking of pip show cantera-minimal output instead of checking pip show cantera output.

I.e. issue isn't reproduced if install Cantera-minimal python interface over Cantera (full) python interface and vice versa. But is presented if Cantera-minimal is installed over previously installed Cantera-minimal.

[speth]

@band-a-prend - I think the issue should be fixed for cantera-minimal as well by da3985a.

[band-a-prend]

@speth
Yes, cantera-minimal python installation is fixed in cantera-2.6.0b2. Thanks!