Requirements installation

[pslacerda]

Hi,

PyMOL deserves a package requirement installation engine for plugins. The following code snippets are the main mechanisms of package installation.

import subprocess

try:
    import numpy as np
    
except ImportError:
    subprocess.check_call(
        [
            "python",
            "-m",
            "pip",
            "--disable-pip-version-check",
            "install",
            "numpy",
        ],
    )

try:
    from vina import Vina
    import meeko
    import openbabel
    import plip

except ImportError:
    subprocess.check_call(
        [
            "conda",
            "install",
            "-y",
            "-c",
            "conda-forge",
            "numpy",
            "meeko",
            "vina",
            "plip",
            "numpy"
        ],
    )

import shutil
import urllib.request
import tempfile


if not shutil.which('scrub.py'):
    with tempfile.TemporaryDirectory() as tempdir:
        zip = "%s/scrubber.zip" % tempdir
        url = "https://github.com/forlilab/scrubber/archive/refs/heads/develop.zip"
        urllib.request.urlretrieve(url, zip)

        subprocess.check_call(
            [
                "python",
                "-m",
                "pip",
                "--disable-pip-version-check",
                "install",
                zip     
            ]
        )
`'`

[ye11owSub]

hey @pslacerda
In this line you will find the python package requirements for Pymol

[ye11owSub]

numpy will always be installed, so all code similar to the following should be removed

try:
    import numpy as np
...

[pslacerda]

@ye11owSub, I forgot to mention that these mechanisms are for plugin requeriments.

Like in the Vina plugin: https://pymolwiki.org/index.php/Vina

[ye11owSub]

@pslacerda
I may not be aware of all the details, but is this not a transitive dependency for all plugins?
A plugin depends on pymol -> pymol depends on numpy

[pslacerda]

Yes, numpy specifically is already shipped with PyMOL by default, however the pip mechanism allow the installation of plugins requeriments from PyPI.org other than numpy.

The snippets are only illustrative, but suggests the workaround for regular pip packages, conda packages (like from conda-forge) and zip packages (like wheel and github zip packages).

[JarrettSJohnson]

A while back one idea that I had for "Package Manager V2" was user-friendlier handling of dependencies like these so users wouldn't have to do much manual work. One of them was to specify them in some sort of text file for PyMOL to read and install from (similar to an environments/requirement that pip and conda uses); or have something that parses the plugin files to create a dependency tree and install missing packages from there. I'm assuming your approach would require changes to the plugins themselves?

[ye11owSub]

In this case, I see two options for installing specific dependencies:

1. https://peps.python.org/pep-0723/#script-type - For each script specify its dependencies in a dedicated section (/// script).
2. Add the dependency section for all scripts to the pyproject.toml file.

[project.optional-dependencies]
scripts = [
  "a==1.2.3",
  "b==4.5.6",
  "c==7.8.9",
]

The installation process will look like: pip install '.[scripts]'

[pslacerda]

@JarrettSJohnson the idea of a better Package Manager is sound.

Maybe requirements may be declared on the single file docstring.

"""
    = vina.py =

    This plugin enables small scale virtual screening with the AutoDock Vina
    software stack. It uses Meeko and Scrubber to prepare molecular ligands,
    and PLIP to analyze the results.
    
    It was tested on PyMOL 3.0 with Python 3.10. Currently supports only
    Linux and probably Mac.

    @author Pedro Sousa Lacerda
    @email [email protected]
    @pip_requires
        meeko
        https://github.com/forlilab/scrubber/archive/refs/heads/develop.zip
    @conda_requires
        conda-forge::vina>=3
        openbabel==3.1.1
"""

Or maybe install by function calls declared on the code preceding the import. This approach works for single- and multi-file plugins with not much noise in the code.

from pymol import cmd as pm

pm.pip_requires([
    "meeko==0.6.1",
    "https://github.com/forlilab/scrubber/archive/refs/heads/develop.zip"
])
pm.conda_requires([
        "conda-forge::vina>=3",
        "openbabel==3.1.1"
])

@ye11owSub
These approaches are similar to PEP 273 and to inline script metadata in the sense that dependencies are declared on the single file plugin script. The project.toml is nice but would only works on multi-file plugins.

[ye11owSub]

would only works on multi-file plugins

What do you mean by that? Why?

[pslacerda]

would only works on multi-file plugins

What do you mean by that? Why?

pyproject.toml is not Python, is TOML. The plugin would need multiple files with a minimum of 2 (1 Python and 1 TOML).

[pslacerda]

How proposed solutions resolve conflicts? I have no idea.

Even if all requirements from all plugins are requested to pip or conda on a single call, conflicts could happens.

[pslacerda]

@JarrettSJohnson, the require directives could check if is a clean install and fail otherwise.

Edit: It isn't a solution for the function call solution because it could upgrade to unwanted versions. All previously installed plugins requirements must be analyzed together.

[ye11owSub]

@pslacerda

These approaches are similar to PEP 273 and to inline script metadata in the sense that dependencies are declared on the single file plugin script. The project.toml is nice but would only works on multi-file plugins.

It's the same thing. This approach will only work if you want to run a script separately from everything else, like a "one time run"

pyproject.toml is not Python, is TOML. The plugin would need multiple files with a minimum of 2 (1 Python and 1 TOML).

How proposed solutions resolve conflicts? I have no idea.
Even if all requirements from all plugins are requested to pip or conda on a single call, conflicts could happens.

You could store all the dependencies in the pyproject.toml file (in a separate section called scripts) of the pymol-open-source repository. Since pymol is a necessary dependency for all scripts, this approach will work.

[pslacerda]

It's the same thing. This approach will only work if you want to run a script separately from everything else, like a "one time run"

They're very similar indeed despite I prefer my docstring approach because looks prettier. However many single file PyMOL plugins (in the sense that extends PyMOL via cmd.extend) could benefit from it... including my plugin I cited previously on this thread.

You could store all the dependencies in the pyproject.toml file (in a separate section called scripts) of the pymol-open-source repository. Since pymol is a necessary dependency for all scripts, this approach will work.

pyproject.toml works only for packages and not for modules, requiring an additional mechanism to handle the single module plugin case. Another divergent idea I have from your proposal is that the scripts section from a pyproject.toml seems semantically reserved for scripts and not for the main code, finally the pip .[scripts] syntax applies only to optional set of requirements for the package and the . is only for locally available packages (not ones from remote repositories like PyPI).

This approach you stated is the one i used on the DRUGpy plugin, however the install is slightly different.

[ye11owSub]

They're very similar indeed despite I prefer my docstring approach because looks prettier.

I mean, it's the same thing. We both provided a link to PEP 723.

pyproject.toml works only for packages and not for modules, requiring an additional mechanism to handle the single module plugin case.

my idea is:
Pymol is a package -> Each script in the Pymol-script-repo relies on Pymol -> Use Pymol pyproject.toml to store dependencies for Pymol-script-repo

Another divergent idea I have from your proposal is that the scripts section from a pyproject.toml seems semantically reserved for scripts and not for the main code

You can name this group of dependencies whatever you want. "scripts" sounds like a suitable name for this.

reserved for scripts and not for the main code

pip install '.[scripts]' will install the main code and additional dependencies.

finally the pip .[scripts] syntax applies only to optional set of requirements for the package and the . is only for locally available packages (not ones from remote repositories like PyPI)

I'm using the pip install . command for the sake of simplicity. To install from PyPI, you need to specify the package name. Assuming that pymol is published on PyPI, dependencies can be installed using pip install pymol[scripts].

The following code snippets are the main mechanisms of package installation.

I think none of these examples would be appropriate. You should not force a user to update, install, or remove anything during runtime.

[pslacerda]

They're very similar indeed despite I prefer my docstring approach because looks prettier.

I mean, it's the same thing. We both provided a link to PEP 723.

They're similar but are different. Despite being uglier and already have a tool fully developed, PEP 723 requires more character typing and don't support Anaconda/Mamba.

my idea is: Pymol is a package -> Each script in the Pymol-script-repo relies on Pymol -> Use Pymol pyproject.toml to store dependencies for Pymol-script-repo

Most PyMOL-Scripts-Repo don't require third-party packages. Also they aren't "one time run" scripts, but plugins extending PyMOL with cmd.extend. I mean that plugins from that repository don't only requires PyMOL like a script, but extends it enabling users to do more than PyMOL alone.

The following code snippets are the main mechanisms of package installation.

I think none of these examples would be appropriate. You should not force a user to update, install, or remove anything during runtime.

The user isn't being forced, he/she's installing a given plugin along with its dependencies.