Changing pycv into a proper python package

[Iximiel]

Description

This should solve #1133, the TL;DR is "calling pycv from the python implementation gives problems"

I rewrote the building part of pyCV to look like a python package.

To install it you need plumed available in your path and you just need to use

pip install .

in the pycv dir.

To use it, it is slightly less straightforward (these solution do not work if you installed with pip install -e .):

calling plumed from the shell

When you install with pip you get an extra pycv executable in the bin of the environment. Calling it will print on the shell the path of the .so that contains the pycv actions.
You can copy-paste that path to your plumed.dat and you are ready to go.

calling plumed from python

If you are using plumed in a python script you'll need to have pycv available in the python environment. Then you can get the pycv object to load with:

from pycv import getPythonCVInterface
from plumed import Plumed
...
plmd = Plumed()
...
plmd.cmd("readInputLine",f"LOAD FILE={getPythonCVInterface()}")
...

---

I need to

• refine this to compile 100% correctly (aka write a "findPLUMED.cmake")
• find if there is a more robust way of passing the path
• update the readme
• rewrite its part of the CI (both for the plumed-that-calls-python tests and for the python-calling-plumed-that-calls-python tests)
• update/add new tests
• have some feedback on the user-friendliness
• have some feedback on the names

---

A good number of important projects use scikit-build-core, among them LAMMPS and Cmake, so it is a project destined to remain.

---

This will unlikely pass the CI for the first few commits

@GiovanniBussi @tonigi, what do you think?

Target release

I would like my code to appear in release 2.10 or 2.11?

Type of contribution

• changes to code or doc authored by PLUMED developers, or additions of code in the core or within the default modules
• changes to a module not authored by you
• new module contribution or edit of a module authored by you

Copyright

• I agree to transfer the copyright of the code I have written to the PLUMED developers or to the author of the code I am modifying.

• the module I added or modified contains a COPYRIGHT file with the correct license information. Code should be released under an open source license. I also used the command cd src && ./header.sh mymodulename in order to make sure the headers of the module are correct.

Tests

• I added a new regtest or modified an existing regtest to validate my changes.
• I verified that all regtests are passed successfully on GitHub Actions.

[Iximiel]

It works, but I do not understand and I cannot reproduce locally the segmentation faults that are present in the fedora CI.

[carlocamilloni]

@Iximiel @tonigi @GiovanniBussi is this ready to be merged?

[Iximiel]

no, @tonigi found some problems on mac, there is the mystery of fedora and the readme is updated to the previous version

[tonigi]

Even though liked the "standalonecompile" idea, pip install seems to work better (e.g. in conda). I've pushed updates to the README. For what regards Mac, I put a note here https://github.com/tonigi/plumed2/blob/47e9b7128c73b3fa64d03cb66af8a2220c0dbe43/plugins/pycv/README.md?plain=1#L91 , which is hopefully sufficient to make it work.

[tonigi]

In other words, ok for me to merge (after merging my changes to the readme).

[carlocamilloni]

so i think that once the conflict and the readme are done i can merge it, correct @Iximiel ?

[tonigi]

Something unrelated that would be nice, IMHO, could be to process the plugins with doxygen and add them to the manual (perhaps under a "Plugin" section).

[Iximiel]

I just squashed everything and rebased to v2.10.

I need to update the readme with the @tonigi comment and with an installation guide

And check if the CI on fedora had changed idea and now pass

[Iximiel]

Something unrelated that would be nice, IMHO, could be to process the plugins with doxygen and add them to the manual (perhaps under a "Plugin" section).

Yep, but that is for another PR

As now I am adding to the manual an example to create the documentation for the package with the built-in python tools (menaning with a discutible old-fashioned graphic style)

[Iximiel]

the fedora failures might be due to the python version being 3.12? Shall I comment the test temporarily under the rug @carlocamilloni @GiovanniBussi?

Looking at the logs i cann see that Rocky uses an ancient 3.6 (? o_O)

I forgot to specify that the extra tests I added, the ones that run with pytest, test pycv with the plumed python module

[carlocamilloni]

@Iximiel if it is python 12 the problem than it should be easy to reproduce and it would be better to fix it because isn't that the current supported version? otherwise I am happy to merge it and turn that test off for pycv

[Iximiel]

I'll try from the wsl if I can reproduce that

[tonigi]

I'm trying to rebuild the fedora dockers to play with. I was hoping it somehow maybe related to python embedded interpreters not honoring their environment paths, hence libpython version mixups.

[Iximiel]

The crashing tests are to python+plumed+pycv ones, the other, the plumed+pycv look fine

And also on my pc with 3.12.7, like on fedore is segfaulting in the first line here:

def create_plumed_var(plmd: Plumed, name: str, command: str):
    #command="PYCVINTERFACE ATOMS=@mdatoms IMPORT=pycvPerFrameSTR CALCULATE=pydist PREPARE=changeAtom"
    plmd.cmd("readInputLine", name + ": " + command)
    shape = np.zeros(1, dtype=np.int_)
    plmd.cmd("getDataRank " + name, shape)
    data = np.zeros((1))
    plmd.cmd("setMemoryForData " + name, data)
    return data

If you are ok I would put this as a known bug and temporarly comment that part of the tests pyenv, then I need to work on this, since looks like it is failing with python 3.10 and so on

[tonigi]

Ok for me. BTW note that I introduced a problem in committing the "scripts" symlink. It should have been relative, and its creation removed from the dockerfile (now fixed).

[tonigi]

Locally, docker-building fedora39 and fedora39-pycv is successful...

[tonigi]

Just an hypothesis: the plumed:fedora39 image pushed on the docker repositories is "bad", and it is not being replaced on our test builds