Out-of-tree builds can't find lib SofaComponentGeneral

[marques-bruno]

When building SofaPython3 as out-of-tree build, linker can't find SofaComponentGeneral.
Though, CMAKE_PREFIX_PATH is properly set to SOFA's install dir, SofaPython3's CMakeLists.txt performs the SofaGeneral findPackage and target_link_libraries does include SofaComponentGeneral.

[jnbrunet]

You need to add find_package(SofaComponentGeneral REQUIRED)

I added it to my PR #136 as a quick fix

[ngvpho]

Hello,
I built plugin.Sofapython3 by out-of-tree build, however, in the sofa_root build has only two files:

And I cannot load sofa_python3.

Could you please help me to solve this problem?

[damienmarchal]

Hi Pho,
Can you give us more details on which sofa version you are using for the out of tree build ?
Is this a pre-made binary version that we ship or your own ? On my side not sure SofaPython3 can be build out of tree with something other than an up to date sofa build.

[ngvpho]

Hi Damien,
I installed Sofa v19.06, which got from this link in sofa build:
git clone -b v19.06 https://github.com/sofa-framework/sofa.git sofa/src/
The errors are as attached file bellow:
Error in installing sofapython3.pdf

[jnbrunet]

Hi Pho,

@damienmarchal is right, you will need to build the development version of SOFA as SofaPython3 is still isn't released yet.

Before starting, let's define some environment variables that will help us:

$ export SOFA_SRC=$HOME/sources/sofa
$ export SOFA_BLD=$HOME/sources/sofa/build

Here SOFA_SRC is the path to the directory of SOFA's source code (this is the path that was created by the git clone command), and SOFA_BLD is the path to the directory where SOFA was built (this is the path where the make command is done).

You can do the same thing for SofaPython3 source code and built directories.

$ export SP3_SRC=$HOME/sources/plugin.SofaPython3
$ export SP3_BLD=$HOME/sources/plugin.SofaPython3/build

(Make sure those paths are good)

1. Building the development version of SOFA

Checkout the master branch of sofa and build it

$ cd $SOFA_SRC
$ git checkout master
$ git pull
$ cd $SOFA_BLD
$ cmake $SOFA_SRC
$ make
$ make install

(The last step make install is very important here, it should install SOFA's binaries in the directory $SOFA_BLD/install.)

2. Building the plugin SofaPython3

It is in this step that we need to give CMake the path to SOFA's installation path of the last step.

$ cd $SP3_SRC
$ git pull
$ cd $SP3_BLD
$ cmake -DCMAKE_PREFIX_PATH=$SOFA_BLD/install/lib/cmake $SP3_SRC
$ make

At this point, make sure you can find the file $SP3_BLD/lib/libSofaPython3.so. We now need to install the SofaPython3 python package into the python's user site directory:

$ mkdir -p $(python3 -m site --user-site)

Linux

$ ln -sFfv $(find $SP3_BLD/lib/site-packages -maxdepth 1 -mindepth 1 -not -name "*.py") $(python3 -m site --user-site)

Mac OSX

$ ln -sFfv $(find $SP3_BLD/lib/site-packages ! -name *.py -d 1) $(python3 -m site --user-site)

3. Running SOFA with the plugin

Finally, launch SOFA with

$ ./$SOFA_BLD/bin/runSofa -l $SP3_BLD/lib/libSofaPython3.so

4.. Running SOFA directly in python

You should also be able to load SOFA directly into your python 3 interpreter:

$ python3

>>> import SofaRuntime
>>> import Sofa
>>> SofaRuntime.importPlugin('SofaComponentAll')
>>> root = Sofa.Core.Node("root")
>>> root.addObject("MechanicalObject", position=[0, 0, 0, 5, 5, 5])

[damienmarchal]

Thank you for pointing the weakness of the out-of-source build documentation.

So here is a more detailled one that I just tested and that work.

Sofa

1. get the sofa source code using git to clone the master branch from https://github.com/sofa-framework/sofa.git sofa/src/
   type:

git clone https://github.com/sofa-framework/sofa.git sofa/src/ 

2. compile sofa following the video tutorial. At this step, please test that you have a runSofa application in the "bin" directory. If this is not the case no need to go further.

3. once this is done in the sofa build directory do an install step

make install

or with ninja (if you use it)

ninja install

Now you should have a directory called "install"

/home/ho-lab/Documents/MySofa/P-sofa/build/install

At this point you are ready to make an out-of-tree build of the python plugin.

SofaPython3

You can keep the organization of your directory (the one you are specifying in the pdf file you send). With SofaPython3 next to the P-sofa folder.

So

1. get the SofaPython3 source code using git to clone the master branch.

git clone [email protected]:SofaDefrost/plugin.SofaPython3.git 

2. use cmake to configure the build script specifying the CMAKE_PREFIX_PATH to point to the location of your sofa installation. So in your case it should be

CMAKE_PREFIX_PATH=/home/ho-lab/Documents/MySofa/P-sofa/build/install

When you click on generate you shouldn't have error/failure and you normally don't need to specify the PYTHON_ROOT_DIR or Python_EXECUTABLE. This only happens if you have already compiled your sofa version with python2.7 enabled. So by default don't do that. At this step you now can compile the python plugin.

3. compile the plugin by going into your SofaPython3 build directory and type
   ninja or make depending on the build system you use.

When the compilation is done you should have in the SofaPython3 build directory a subdirectory called "lib" with this content: libSofaPython3.so libSofaPython3.so.1.0 site-packages.

[damienmarchal]

Haha :@jnbrunet & I responded at he same time ;)
Thanks @jnbrunet

But Jean Nicolas, on my system If I start runSofa and try to load the plugin loaded it fails to load because the site-packages is not yet registered. So I assume that one additional step is needed to copy the configuration file from the python3 etc/ directory into the sofa one ?
Is there a different appraoch ?

[jnbrunet]

Damn @damienmarchal , 1 minute between our replies!

Indeed, I forgot about the python package. There are many ways to do it, mine is:

4. Install the SofaPython3 python package into your python's user site directory
   ln -sFfv $(find $SP3_BLD/lib/site-packages ! -name *.py -d 1) $(python3 -m site --user-site)
   This will create symbolic links, so you only need to do it once after the compilation

I'll edit my answer to add this step

[ngvpho]

Thank you so much @jnbrunet and @damienmarchal
Now i can run sofa python 3 by using command: ./$SOFA_BLD/bin/runSofa -l $SP3_BLD/lib/libSofaPython3.so. The scene appear normally:

However when loading a .py file it appeared errors as my attached file:

"The sofa cannot detect .py file"

Could you please help me to solve it?

[damienmarchal]

Hi Pho,

There is an [Error] message saying that it cannot load /lib/SofaPython3.so

When you are typing:

/$SOFA_BLD/bin/runSofa -l $SP3_BLD/lib/libSofaPython3.so

Did you set the environment variable $SP3_BLD before ? Otherwise it is empty, so it would not be surprising that the plugin does not load.

If you don't know how to do this in Linux here is a starting point:
https://www.serverlab.ca/tutorials/linux/administration-linux/how-to-set-environment-variables-in-linux/

DM.

[ngvpho]

Hi Damien,
Many thanks for your quick reply. Actually, I already set $SP3_BLD, and in lib folder also includes 3 files you required: libSofaPython3.so libSofaPython3.so.1.0 site-packages

Now I type: ln -sFfv $(find $SP3_BLD/lib/site-packages ! -name *.py -d 1) $(python3 -m site --user-site), it warned as follows:

Then, I types: ./$SOFA_BLD/bin/runSofa -l $SP3_BLD/lib/libSofaPython3.so, it appears error:

"pybind11::error_already_set" ? I already install pybind11 in this image:

Pho

[jnbrunet]

Hey Pho,

My bad, I was on Mac osx when testing.

Try with

ln -sFfv $(find $SP3_BLD/lib/site-packages -maxdepth 1 -mindepth 1 -not -name "*.py") $(python3 -m site --user-site)

instead.

[ngvpho]

Hi Jnbrunet,
Thank you for your answer.
I tried with your suggestion, it appears error:
.
Actually, there is no site-packages folder in this path: ./lib/python3.7/..

I tried to install site-packages for python3 in many methods, it errors:

[jnbrunet]

Then simply create it...

$ mkdir -p $HOME/.local/lib/python3.7/site-packages

site-packages isn't a package. It is the directory where python looks to find packages.

[ngvpho]

Hi Jnbrunet and Damien,
Thank you so much for your help. Now i can open sofapython3.
Pho

[ngvpho]

Hello,
When adding file plugin_list.conf.default in ~/Documents/Pho.Nguyen/MySofa/sofa/build/lib, I cannot cannot loading plugin SP3:

When remove that file, I also cannot open sofapython3.
What should I do in this case?

[damienmarchal]

Can you provide more details (screenshot) on the two results because I d'ont understand yet what you are saying;

[ngvpho]

Hi Damien,
Last time, after building plugin sofapython3, I had to detele the file: "plugin_list.conf.default" in this folder

and I can load the soft-eel.py file. Today, the errors happened:
if delete the file: "plugin_list.conf.default", error is as follows:

and in case I keep the file: "plugin_list.conf.default", error is:

[jnbrunet]

hey @ngvpho

Do not delete the file plugin_list.conf.default.

It seems you are missing the ".so" extension to your plugin. Change

$ ./runSofa $SP3_BLD/lib/libSofaPython3

for

$ ./runSofa $SP3_BLD/lib/libSofaPython3.so

Next, your are loading the the old SofaPython plugin. This plugin is incompatible with the plugin SofaPython3. To remove it, first unset the SofaPython option in the compilation of Sofa.

$ cd $SOFA_BLD
$ cmake-gui .

and uncheck the PLUGIN_SOFAPYTHON option.
Rebuild Sofa

$ make install

And make sure the SofaPython plugin isn't in the file $SOFA_BLD/config/loadedPlugins.ini

[ngvpho]

Hi @jnbrunet,
After following your suggestion, it still errors:

These folders have:

[damienmarchal]

I think the -l is missing in:

$ ./runSofa $SP3_BLD/lib/libSofaPython3.so

Shouldn't it be:

$ ./runSofa -l $SP3_BLD/lib/libSofaPython3.so

[ngvpho]

Hi @damienmarchal and @jnbrunet ,
Thank you so much for your help. Now it works.^^

[jnbrunet]

Alright, I think we can close this issue now. The initial bug of this issue was fixed by #136 .

[ngvpho]

Hi,

Last time, I successfully installed sofa python3 in ubuntu19.10. Recently, I am installing sofa python3 on another computer running in ubuntu20.04. The configuration and compile were fine without any problems. But, it lacked file libSofaPython3.so.01 in $SP3_BLD/lib/. See following image:

and then, when running "./runSofa -l $SP3_BLD/lib/libSofaPython3.so", it appeared main screen, and detect .py file, but cannot load them. See following image:

I don't know why Sofa cannot load "plugin Sp3".
Anyone can help me?

Best,
Pho

[damienmarchal]

Hello @ngvpho,

Thanks for reporting the problem.

Instead of loading the plugin with the -I option in the command line can you load it manually using the menu "View/PluginManager..." ? Then try to load the scene.

In that case does it work ? If it is not the case then maybe it is a bug somewere that we need to investigate. @jnbrunet does it ring a bell to you ?

[jnbrunet]

Hi @ngvpho ,

It cannot find it because it isn't in the SOFA's plugins directory ($SOFA_ROOT/plugins) nor the SOFA_PLUGIN_PATH environment variable. It finds it the first time with the runSofa -l , but at the line requiredPlugin(SofaPython3) SOFA tries to find it a second time (without even verifying that it is already loaded...), and cannot find it.

Since you already loaded the plugin with the runSofa's -l option, normally you don't have to add it a second time with the requiredPlugin(SofaPython3), so you could remove the line from your scene.

[damienmarchal]

In general at Defrost we recommand to always put all the required plugin in every scenes. Doing so clearly states what are the needs for the scene to run and makes them much more robust (and sharable) because they makes no assumption on how is configure/initialized the execution environment.

If doing so rises a warning/error message then I think we should fix this behavior instead of fixing the scene. But I'm surprise this the case ?

[jnbrunet]

Hi @damienmarchal ,

Of course. However, here you need the plugin to be able to load the scene (.py). In this case, @ngvpho explicitly stated the complete path to the plugin via the -l option so that runSofa can load its python scene. But this path isn't automatically added to the plugin search paths of SOFA.

[damienmarchal]

@jnbrunet it is unclear to me if you think:

• the error message is legitimate and thus nothing has to be done
• the error message is not legitimate and if we should fix the plugin manager so that it stop printing that.

[jnbrunet]

Hey @damienmarchal ,

The error message is legitimate: you asked SOFA to find the plugin SofaPython3 via a requiredPlugin component, but it cannot find it. SofaPython3 has been compiled out-of-source and the build directory of the plugin isn't in SOFA's plugin paths.

SOFA has no way to tell if the plugin is already loaded (in this case, via the -l option) since it does its comparison between plugins by first loading them in memory and getting the result of the global method getModuleName(), usually defined in the init.cpp file of the plugin.

[damienmarchal]

Ok, on my side I don't think RequiredPlugin is equivalent to FindPlugin neither LoadPlugin, it is just saying it is required and consequently if it is already "there", the requierement is fullfill and I see no good reason why to should print an error.

About the fact that "SOFA has no way to tell if the plugin is already loaded"...
Shouldn't the existing PluginManage::isPluginLoaded() and/or the PluginManager::m_pluginMap may be of some help there ?

EDIT: If I have some time I will try to make some tests to see if it is doable.

[jnbrunet]

Unfortunately, the PluginManage::isPluginLoaded() doesn't exactly do this. It takes as an argument a plugin path, than tries to load it and compare the paths of both plugins:

bool PluginManager::pluginIsLoaded(const std::string& plugin)
{
    std::string pluginPath = plugin;

    if (!FileSystem::isFile(plugin)) {
        pluginPath = findPlugin(plugin);
    }

    return m_pluginMap.find(pluginPath) != m_pluginMap.end();
}

I agree that RequiredPlugin(plugin_name) should check if a loaded plugin has its "name" (ie getModuleName()) equals to "plugin_name". This is could be a useful PR on SOFA's side, but not related to SP3 however.

[ngvpho]

Hi @damien,

I can find SofaPython3 in plugin manager

Hi @jnbrunet

I run the command again without -l, but it is still error.

Do you have any ideas?
Best,
Pho

[jnbrunet]

Hey @ngvpho ,

The "-l" parameter expects a path to a plugin library (.so), example runSofa -l libSofaPython3.so. If you remove the "-l" but let the path there, than SOFA will see this path as a scene to load (ex runSofa libSofaPython3.so, which is wrong here, libSofaPython3.so is not a scene but a plugin, which is why you are having this error. Simply remove the path from the command line arguments unless you prefix it with the "-l".

JN

[ngvpho]

Hi @jnbrunet,

Ok, I can load it.
Thanks for your help. ^^

Best,
Pho