Wrapper Update #727

varunagrawal · 2021-03-24T04:57:30Z

Updated the wrapper so we have all the new goodies!

3eff76f60 Merge pull request #53 from borglab/feature/refactor 13215dfa7 Merge pull request #52 from borglab/fix/tests 696913b11 install setuptools 9523328ba Merge branch 'master' into fix/tests 7c630b361 some more cleanup 656993a71 cleaned up Typename a16f6f38e move qualified and basis type outside to their own class scope 72ead8bd7 Merge pull request #51 from borglab/fix/test-interface-parser 6deefd4fc added tests for interface_parser 50d490a85 Merge pull request #50 from borglab/feature/refs-all-types be4511290 updated docs for BasisType 0e80b0d8c update MATLAB tests 0015d7397 added support for shared pointer and ref for basis types 86d2158f1 remove std::string from list of Basis types 94f928441 ignore code coverage reports 2033dd345 replace prints with log.debug statements ae98091b3 fix deprecation in doc tests 13a2f66c4 Merge pull request #46 from borglab/feature/new-shared-pointer 3c7d85865 updated docs 6d7897088 use @ for raw pointer, go back to * for shared pointer 1d6194c57 updated matlab wrapper to handle both raw and shared pointers 1448f6924 fix some failing tests 2ab1dae32 Merge branch 'master' into feature/new-shared-pointer 96f8a56bd Merge pull request #47 from borglab/fix/ci 6003203f3 run CI only for pull requests a8f29ead1 fix the python version yml key fcae17227 check if directory exists when testing f592f08c9 explicit pip3 so that we don't use Python2 d49c2f3c2 correct call for pip dfe360526 fix the CI 149b7c523 docs for templated functions f2189acc6 support typedefing for templated functions 965458a2b added test for templated functions eaff6e6ab made is_const common for all types 3d9c70b32 added tests and cleaned up a bit 010b89626 support for simple pointers on basis types 6b98fd80c new syntax for shared_ptr ff7ad0b78 support for templated functions a1a443c8d Merge pull request #43 from borglab/fix/cmake-and-matlab 2f3a055e4 remove accidentally committed file 770d055e2 set proper paths for cmake and eschew relative paths 773d01ae1 fix bug in matlab wrapper 721ef740f Merge pull request #41 from borglab/feature/type-hints 67aac9758 minor refactor of CI yml e6a63ae0c fix all mypy issues a3aaa3e7c remove a lot of linter issues from matlab_wrapper a96db522f static typing for interface_parser git-subtree-dir: wrap git-subtree-split: 3eff76f604b5ba9e71cf4947654e288142ed7a94

varunagrawal · 2021-03-24T04:58:11Z

@carlmorgan can you please use this branch to check for yourself?

carlmorgan · 2021-03-24T09:19:33Z

@carlmorgan can you please use this branch to check for yourself?

Off the bat:

<HOME>/projects/git/thirdParty/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:1:10: fatal error: gtwrap/matlab.h: No such file or directory
    1 | #include <gtwrap/matlab.h>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [matlab/CMakeFiles/gtsam_matlab_wrapper.dir/build.make:77: matlab/CMakeFiles/gtsam_matlab_wrapper.dir/__/wrap/gtsam/gtsam_wrapper.cpp.o] Error 1

With the generator patch / hack from my original issue post, this then generates #include <wrap/matlab.h> which all compiles / completes OK.

For running the tests, I had to modify my Matlab libstdc++ to use the ubuntu base system one (but this step maybe a legacy of previous hacking on my local machine)

cd /usr/local/MATLAB/R2020b/sys/os/glnxa64/
sudo mv libstdc++.so.6.0.25 libstdc++.so.6.0.25_matlab 
sudo cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28 libstdc++.so.6.0.28_ubuntu
sudo ln -sf libstdc++.so.6.0.28_ubuntu libstdc++.so.6

Then running the tests

cd target_install/gtsam_toolbox/
export LD_LIBRARY_PATH="../lib:/usr/lib:/usr/local/lib"
/usr/local/bin/matlab" -r "addpath gtsam_tests ; test_gtsam ; quit" -nodisplay -nojvm

                                                    < M A T L A B (R) >
                                          Copyright 1984-2020 The MathWorks, Inc.
                                      R2020b Update 1 (9.9.0.1495850) 64-bit (glnxa64)
                                                     September 30, 2020


For online documentation, see https://www.mathworks.com/support
For product information, visit www.mathworks.com.

Starting: testCal3Unified
Starting: testKalmanFilter
Starting: testJacobianFactor
Starting: testValues
Starting: testPriorFactor
Starting: testLocalizationExample
Starting: testOdometryExample
Starting: testPlanarSLAMExample
Starting: testPose2SLAMExample
Starting: testPose3SLAMExample
Starting: testSFMExample
Starting: testStereoVOExample
Starting: testVisualISAMExample
Starting: testUtilities
Starting: testSerialization
Tests complete!

dellaert · 2021-03-24T16:14:41Z

@mksoccer Is also interested in having a working MATLAB wrapper. Adding him on this issue.

548e61b1f Merge pull request #57 from borglab/fix/configurable-matlab-include b58eabaf1 set correct template file path 483cdab9c fix 1f393516d fix CI syntax 8f0a3543f more concise cmake command because we don't care about the extra files generated 641ad1326 update CI to run cmake de6b9260f added CMake variable to configure the include directory for matlab.h cbe5f18bc Merge pull request #54 from borglab/feature/refactor2 cc78ee3bb test formatting 046a50b01 break down interface_parser into a submodule of smaller parts git-subtree-dir: wrap git-subtree-split: 548e61b1fbf02759d2e4a52435c2f1b3cbde98f0

29628426d Merge pull request #59 from borglab/fixes a95429ea0 Merge pull request #56 from borglab/fix/this-instantiation 3e22d6516 more documenatation and some formatting 526301499 updated the test to test for non-templated This cdb75f36f Merge branch 'master' into fix/this-instantiation 0f5ae3b7f moved example pybind template to templates directory d55f5db38 remove extra whitespace 21891ad3d skip tests until we figure out what's going on 2ea6307c3 better way of handling the matlab includes in the matlab wrapper d0f8a392c Merge pull request #55 from borglab/feature/refactor3 57d47cbd9 create directories to store generated output 4788a1e37 fixed This instantiation 61d2cbfc4 add namespace test to matlab wrapper ec39023e6 added more, smaller tests for Python wrapper 19c35b857 test for matlab class inheritance and some clean up 06ca5da13 test for matlab functions cb05d7379 minor clean up and separate tests for geometry and class 8d8145cc4 break down test interface file into smaller files that can be easily debugged 97328f057 restructured test files and added dedicated fixtures directory git-subtree-dir: wrap git-subtree-split: 29628426d2c1a7bb728e40307c0f25cb468cd1bc

varunagrawal · 2021-03-25T04:52:39Z

@carlmorgan I believe I have figure out the include file issue.

carlmorgan · 2021-03-25T07:40:36Z

@carlmorgan I believe I have figure out the include file issue.

Built and passed tests (both 04b9625 and 82d6e68) for us, BIONIC and FOCAL (w/Matlab 2020b1) - I will try and run a few other configs if I can this evening, but all looks good.

carlmorgan · 2021-03-25T11:53:16Z

Misc testing (with no depth of understanding) - let me know if you want separate issues / or feel free to move them:

With Python bindings enabled, wrap/scripts/pybind_wrap.py is not executable, needed chmod 755 for it to continue.

<HOME>/gtsam/target_build$ cd <HOME>/gtsam/target_build/python && /usr/bin/cmake -E env PYTHONPATH=<HOME>/gtsam/wrap/cmake/..: <HOME>/gtsam/wrap/scripts/pybind_wrap.py --src <HOME>/gtsam/gtsam/gtsam.i --out gtsam.cpp --module_name gtsam --top_module_namespaces gtsam --ignore gtsam::Point2 gtsam::Point3 gtsam::LieVector gtsam::LieMatrix gtsam::ISAM2ThresholdMapValue gtsam::FactorIndices gtsam::FactorIndexSet gtsam::IndexPairSetMap gtsam::IndexPairVector gtsam::BetweenFactorPose2s gtsam::BetweenFactorPose3s gtsam::Point2Vector gtsam::Point3Pairs gtsam::Pose3Pairs gtsam::Pose3Vector gtsam::KeyVector gtsam::BinaryMeasurementsUnit3 gtsam::KeyPairDoubleMap --template <HOME>/gtsam/python/gtsam/gtsam.tpl --use-boost
Permission denied

I think there might also be issues with GTSAM_UNSTABLE_INSTALL_MATLAB_TOOLBOX:BOOL=TRUE,

If I include with
cmake CMAKE_ARGS=-DGTSAM_USE_SYSTEM_EIGEN:BOOLEAN=FALSE -DGTSAM_WITH_EIGEN_MKL:BOOLEAN=FALSE -DGTSAM_WITH_TBB:BOOLEAN=FALSE -DGTSAM_BUILD_EXAMPLES_ALWAYS:BOOL=FALSE -DGTSAM_BUILD_UNSTABLE:OPTION=ON -DGTSAM_USE_QUATERNIONS:BOOL=TRUE -DGTSAM_BUILD_TESTS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DGDB:BOOL=TRUE -DDEBUG:BOOLEAN=TRUE -DASSERT:BOOLEAN=TRUE -DASSERT2:BOOLEAN=TRUE -DGTSAM_BUILD_PYTHON:BOOL=TRUE -DGTSAM_UNSTABLE_INSTALL_MATLAB_TOOLBOX:BOOL=TRUE -DGTSAM_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DMEX_COMMAND:STRING=/usr/local/MATLAB/R2020b/bin/mex ../

Without it - everything goes through / builds / tests fine, with it cmake failures possibly around pthread lib inclusion (which is used elsewhere)

varunagrawal · 2021-03-25T12:24:55Z

That permission denied issue is super weird. There should be no reason for that since we CMake doesn't touch that file other than to invoke it at binding.

Can you share the error message for the unstable toolbox build along with any relevant system info?

varunagrawal · 2021-03-25T12:26:57Z

I'm going to try and reproduce the permission denied issue.

varunagrawal · 2021-03-25T13:37:42Z

I found out the problem with the second issue. Will add the fix to this PR.

@carlmorgan your cmake command really helped me out with this one so you deserve full credit for it.

varunagrawal · 2021-03-25T17:28:00Z

Figured out and fixed the permission denied issue as well. Great catch and reporting @carlmorgan.

9a467794e Merge pull request #61 from borglab/fix/python-variables 6bae7af99 added more status messages and set the PYBIND11_PYTHON_VERSION each time 5129cf3b9 set both sets of Python variables and find python version when including PybindWrap 5a67b526c Merge pull request #60 from borglab/fix/multi-template-methods 4a73b29ef better method names for testing templated methods 989fdf946 added unit test for multi-template methods a56908c21 add namespace qualification to instantiations a25d9631e graceful handling of templated method instantiations 0baa5ab5d multiple templates in methods git-subtree-dir: wrap git-subtree-split: 9a467794e8542872b2782cdaec338aaa30a92e33

varunagrawal · 2021-03-25T22:42:43Z

Okay making some big changes. Plus I broke something on the Matlab wrapper parser so need to figure that out as well.

carlmorgan · 2021-03-26T01:45:49Z

More 'other testing', [ develop + 1e8973e - Focal x86_64 / Matlab2020b1 / Python 3.8.5 ] - will stop until you let me know the other 'big changes' have been resolved.

unstable matlab ON

CMAKE_ARGS=-DGTSAM_USE_SYSTEM_EIGEN:BOOLEAN=FALSE -DGTSAM_WITH_EIGEN_MKL:BOOLEAN=FALSE -DGTSAM_WITH_TBB:BOOLEAN=FALSE -DGTSAM_BUILD_EXAMPLES_ALWAYS:BOOL=FALSE -DGTSAM_BUILD_UNSTABLE:OPTION=ON -DGTSAM_USE_QUATERNIONS:BOOL=TRUE -DGTSAM_BUILD_TESTS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DGDB:BOOL=TRUE -DDEBUG:BOOLEAN=TRUE -DASSERT:BOOLEAN=TRUE -DASSERT2:BOOLEAN=TRUE -DGTSAM_UNSTABLE_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DGTSAM_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DMEX_COMMAND:STRING=/usr/local/MATLAB/R2020b/bin/mex
CXX_FLAGS=-DEIGEN_DONT_VECTORIZE -DEIGEN_DISABLE_UNALIGNED_ARRAY_ASSERT

cd <HOME><DIR>/gtsam/target_build/matlab && /usr/bin/c++  -DBOOST_ALL_NO_LIB -DBOOST_ATOMIC_DYN_LINK -DBOOST_CHRONO_DYN_LINK -DBOOST_DATE_TIME_DYN_LINK -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_REGEX_DYN_LINK -DBOOST_SERIALIZATION_DYN_LINK -DBOOST_SYSTEM_DYN_LINK -DBOOST_THREAD_DYN_LINK -DBOOST_TIMER_DYN_LINK -Dgtsam_matlab_wrapper_EXPORTS -I<HOME><DIR>/gtsam/gtsam/3rdparty/metis/include -I<HOME><DIR>/gtsam/gtsam/3rdparty/metis/libmetis -I<HOME><DIR>/gtsam/gtsam/3rdparty/metis/GKlib -I<HOME><DIR>/gtsam -I<HOME><DIR>/gtsam/target_build -I<HOME><DIR>/gtsam/CppUnitLite -I<HOME><DIR>/gtsam/gtsam/3rdparty/Eigen -isystem <HOME><DIR>/gtsam/gtsam/3rdparty/SuiteSparse_config -isystem <HOME><DIR>/gtsam/gtsam/3rdparty/Spectra -isystem <HOME><DIR>/gtsam/gtsam/3rdparty/CCOLAMD/Include  -DEIGEN_DONT_VECTORIZE -DEIGEN_DISABLE_UNALIGNED_ARRAY_ASSERT -O3 -DNDEBUG -fPIC      "-I/usr/local/MATLAB/R2020b/extern/include" -DMATLAB_MEX_FILE -DMX_COMPAT_32 -march=native -o CMakeFiles/gtsam_matlab_wrapper.dir/__/wrap/gtsam/gtsam_wrapper.cpp.o -c <HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp: In function ‘void gtsamNonlinearFactorGraph_addPrior_1222(int, mxArray**, int, const mxArray**)’:
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:13993:59: error: no matching function for call to ‘gtsam::NonlinearFactorGraph::addPrior<gtsam::PinholeCamera>(size_t&, gtsam::PinholeCamera<gtsam::Cal3Bundler>&, boost::shared_ptr<gtsam::noiseModel::Base>&)’
13993 |   obj->addPrior<gtsam::PinholeCamera>(key,prior,noiseModel);
      |                                                           ^
In file included from <HOME><DIR>/gtsam/gtsam/geometry/triangulation.h:27,
                 from <HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:44:
<HOME><DIR>/gtsam/gtsam/nonlinear/NonlinearFactorGraph.h:225:10: note: candidate: ‘template<class T> void gtsam::NonlinearFactorGraph::addPrior(gtsam::Key, const T&, const SharedNoiseModel&)’
  225 |     void addPrior(Key key, const T& prior,
      |          ^~~~~~~~
<HOME><DIR>/gtsam/gtsam/nonlinear/NonlinearFactorGraph.h:225:10: note:   template argument deduction/substitution failed:
<HOME><DIR>/gtsam/gtsam/nonlinear/NonlinearFactorGraph.h:241:10: note: candidate: ‘template<class T> void gtsam::NonlinearFactorGraph::addPrior(gtsam::Key, const T&, const Matrix&)’
  241 |     void addPrior(Key key, const T& prior, const Matrix& covariance) {
      |          ^~~~~~~~
<HOME><DIR>/gtsam/gtsam/nonlinear/NonlinearFactorGraph.h:241:10: note:   template argument deduction/substitution failed:
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp: In function ‘void gtsamValues_at_1289(int, mxArray**, int, const mxArray**)’:
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:14579:120: error: no matching function for call to ‘gtsam::Values::at<gtsam::PinholeCamera>(size_t&)’
14579 |   out[0] = wrap_shared_ptr(boost::make_shared<gtsam::PinholeCamera<gtsam::Cal3Bundler>>(obj->at<gtsam::PinholeCamera>(j)),"gtsam.PinholeCameraCal3Bundler", false);
      |                                                                                                                        ^
In file included from <HOME><DIR>/gtsam/gtsam/nonlinear/Values.h:551,
                 from <HOME><DIR>/gtsam/gtsam/nonlinear/NonlinearFactor.h:23,
                 from <HOME><DIR>/gtsam/gtsam/slam/TriangulationFactor.h:18,
                 from <HOME><DIR>/gtsam/gtsam/geometry/triangulation.h:26,
                 from <HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:44:
<HOME><DIR>/gtsam/gtsam/nonlinear/Values-inl.h:342:20: note: candidate: ‘template<class ValueType> const ValueType gtsam::Values::at(gtsam::Key) const’
  342 |    const ValueType Values::at(Key j) const {
      |                    ^~~~~~
<HOME><DIR>/gtsam/gtsam/nonlinear/Values-inl.h:342:20: note:   template argument deduction/substitution failed:
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp: In function ‘void gtsamISAM2_calculateEstimate_1554(int, mxArray**, int, const mxArray**)’:
<HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:16983:137: error: no matching function for call to ‘gtsam::ISAM2::calculateEstimate<gtsam::PinholeCamera>(size_t&)’
16983 |   out[0] = wrap_shared_ptr(boost::make_shared<gtsam::PinholeCamera<gtsam::Cal3Bundler>>(obj->calculateEstimate<gtsam::PinholeCamera>(key)),"gtsam.PinholeCameraCal3Bundler", false);
      |                                                                                                                                         ^
In file included from <HOME><DIR>/gtsam/target_build/wrap/gtsam/gtsam_wrapper.cpp:84:
<HOME><DIR>/gtsam/gtsam/nonlinear/ISAM2.h:224:9: note: candidate: ‘template<class VALUE> VALUE gtsam::ISAM2::calculateEstimate(gtsam::Key) const’
  224 |   VALUE calculateEstimate(Key key) const {
      |         ^~~~~~~~~~~~~~~~~
<HOME><DIR>/gtsam/gtsam/nonlinear/ISAM2.h:224:9: note:   template argument deduction/substitution failed:
make[2]: *** [matlab/CMakeFiles/gtsam_matlab_wrapper.dir/build.make:77: matlab/CMakeFiles/gtsam_matlab_wrapper.dir/__/wrap/gtsam/gtsam_wrapper.cpp.o] Error 1
make[2]: Leaving directory '<HOME><DIR>/gtsam/target_build'
make[1]: *** [CMakeFiles/Makefile2:30412: matlab/CMakeFiles/gtsam_matlab_wrapper.dir/all] Error 2
make[1]: Leaving directory '<HOME><DIR>/gtsam/target_build'
make: *** [Makefile:163: all] Error 2

Python ON, Unstable Matlab OFF

Looks like the chmod might have been lost again.

CMAKE_ARGS=-DGTSAM_USE_SYSTEM_EIGEN:BOOLEAN=FALSE -DGTSAM_WITH_EIGEN_MKL:BOOLEAN=FALSE -DGTSAM_WITH_TBB:BOOLEAN=FALSE -DGTSAM_BUILD_EXAMPLES_ALWAYS:BOOL=FALSE -DGTSAM_BUILD_UNSTABLE:OPTION=ON -DGTSAM_USE_QUATERNIONS:BOOL=TRUE -DGTSAM_BUILD_TESTS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DGDB:BOOL=TRUE -DDEBUG:BOOLEAN=TRUE -DASSERT:BOOLEAN=TRUE -DASSERT2:BOOLEAN=TRUE -DGTSAM_BUILD_PYTHON:BOOL=TRUE -DGTSAM_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DMEX_COMMAND:STRING=/usr/local/MATLAB/R2020b/bin/mex
CXX_FLAGS=-DEIGEN_DONT_VECTORIZE -DEIGEN_DISABLE_UNALIGNED_ARRAY_ASSERT

make -f python/CMakeFiles/pybind_wrap_gtsam.dir/build.make python/CMakeFiles/pybind_wrap_gtsam.dir/depend
make[2]: Entering directory '<HOME><DIR>/gtsam/target_build'
cd <HOME><DIR>/gtsam/target_build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" <HOME><DIR>/gtsam <HOME><DIR>/gtsam/python <HOME><DIR>/gtsam/target_build <HOME><DIR>/gtsam/target_build/python <HOME><DIR>/gtsam/target_build/python/CMakeFiles/pybind_wrap_gtsam.dir/DependInfo.cmake --color=
Dependee "<HOME><DIR>/gtsam/target_build/python/CMakeFiles/pybind_wrap_gtsam.dir/DependInfo.cmake" is newer than depender "<HOME><DIR>/gtsam/target_build/python/CMakeFiles/pybind_wrap_gtsam.dir/depend.internal".
Dependee "<HOME><DIR>/gtsam/target_build/python/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "<HOME><DIR>/gtsam/target_build/python/CMakeFiles/pybind_wrap_gtsam.dir/depend.internal".
Scanning dependencies of target pybind_wrap_gtsam
make[2]: Leaving directory '<HOME><DIR>/gtsam/target_build'
make -f python/CMakeFiles/pybind_wrap_gtsam.dir/build.make python/CMakeFiles/pybind_wrap_gtsam.dir/build
make[2]: Entering directory '<HOME><DIR>/gtsam/target_build'
[ 92%] Generating gtsam.cpp
cd <HOME><DIR>/gtsam/target_build/python && /usr/bin/cmake -E env PYTHONPATH=<HOME><DIR>/gtsam/wrap/cmake/..: <HOME><DIR>/gtsam/wrap/scripts/pybind_wrap.py --src <HOME><DIR>/gtsam/gtsam/gtsam.i --out gtsam.cpp --module_name gtsam --top_module_namespaces gtsam --ignore gtsam::Point2 gtsam::Point3 gtsam::LieVector gtsam::LieMatrix gtsam::ISAM2ThresholdMapValue gtsam::FactorIndices gtsam::FactorIndexSet gtsam::IndexPairSetMap gtsam::IndexPairVector gtsam::BetweenFactorPose2s gtsam::BetweenFactorPose3s gtsam::Point2Vector gtsam::Point3Pairs gtsam::Pose3Pairs gtsam::Pose3Vector gtsam::KeyVector gtsam::BinaryMeasurementsUnit3 gtsam::KeyPairDoubleMap --template <HOME><DIR>/gtsam/python/gtsam/gtsam.tpl --use-boost
Permission denied
make[2]: *** [python/CMakeFiles/pybind_wrap_gtsam.dir/build.make:61: python/gtsam.cpp] Error 1
make[2]: Leaving directory '<HOME><DIR>/gtsam/target_build'
make[1]: *** [CMakeFiles/Makefile2:30402: python/CMakeFiles/pybind_wrap_gtsam.dir/all] Error 2
make[1]: Leaving directory '<HOME><DIR>/gtsam/target_build'

git reset --hard gtsam/fix/wrapper-update
HEAD is now at 1e8973ead Merging 'master' into 'wrap'
ls -l wrap/scripts/pybind_wrap.py
-rw-r--r-- 3175 Mar 26 17:32 wrap/scripts/pybind_wrap.py

96ccdfd0b Merge pull request #65 from borglab/fix/special-cases 04c06b7e6 Merge pull request #63 from borglab/fix/cmake bf2c91bd2 fix issue in template instantiation generator 152dbcb12 test for special cases d03004b24 fixed the cmake to discover the correct python version and set all corresponding variables 4cf66e0da Merge pull request #61 from borglab/fix/python-variables 80558e35b added more status messages and set the PYBIND11_PYTHON_VERSION each time 73afd1b0a set both sets of Python variables and find python version when including PybindWrap REVERT: 9a467794e Merge pull request #61 from borglab/fix/python-variables REVERT: 6bae7af99 added more status messages and set the PYBIND11_PYTHON_VERSION each time REVERT: 5129cf3b9 set both sets of Python variables and find python version when including PybindWrap git-subtree-dir: wrap git-subtree-split: 96ccdfd0b84a4dbf1b3e9ed31b95ebc2758be9cc

varunagrawal · 2021-03-26T04:55:13Z

@carlmorgan just pushed the fixes.

carlmorgan · 2021-03-26T05:04:21Z

@carlmorgan just pushed the fixes.

HEAD is now at 92ce070 Merging 'master' into 'wrap'
ls -l wrap/scripts/pybind_wrap.py
-rw-r--r-- 3175 Mar 26 17:58 wrap/scripts/pybind_wrap.py

Still no execute bit - maybe your editor is 'resetting' it?

varunagrawal · 2021-03-26T05:20:53Z

That shouldn't quite matter since we're exclusively calling the python executable on that file

carlmorgan · 2021-03-26T05:58:50Z

That shouldn't quite matter since we're exclusively calling the python executable on that file

As checked in in GIT - fails to build, after chmod builds fine.

cd <HOME><DIR>/gtsam/target_build/python && /usr/bin/cmake -E env PYTHONPATH=<HOME><DIR>/gtsam/wrap/cmake/..: <HOME><DIR>/gtsam/wrap/scripts/pybind_wrap.py --src <HOME><DIR>/gtsam/gtsam/gtsam.i --out gtsam.cpp --module_name gtsam --top_module_namespaces gtsam --ignore gtsam::Point2 gtsam::Point3 gtsam::LieVector gtsam::LieMatrix gtsam::ISAM2ThresholdMapValue gtsam::FactorIndices gtsam::FactorIndexSet gtsam::IndexPairSetMap gtsam::IndexPairVector gtsam::BetweenFactorPose2s gtsam::BetweenFactorPose3s gtsam::Point2Vector gtsam::Point3Pairs gtsam::Pose3Pairs gtsam::Pose3Vector gtsam::KeyVector gtsam::BinaryMeasurementsUnit3 gtsam::KeyPairDoubleMap --template <HOME><DIR>/gtsam/python/gtsam/gtsam.tpl --use-boost

PYTHONPATH is set, but no python executable is specified on the command line.

carlmorgan · 2021-03-26T06:08:24Z

CMAKE_ARGS=-DGTSAM_USE_SYSTEM_EIGEN:BOOLEAN=FALSE -DGTSAM_WITH_EIGEN_MKL:BOOLEAN=FALSE -DGTSAM_WITH_TBB:BOOLEAN=FALSE -DGTSAM_BUILD_EXAMPLES_ALWAYS:BOOL=FALSE -DGTSAM_BUILD_UNSTABLE:OPTION=ON -DGTSAM_USE_QUATERNIONS:BOOL=TRUE -DGTSAM_BUILD_TESTS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DGDB:BOOL=TRUE -DDEBUG:BOOLEAN=TRUE -DASSERT:BOOLEAN=TRUE -DASSERT2:BOOLEAN=TRUE -DGTSAM_BUILD_PYTHON:BOOL=TRUE -DGTSAM_UNSTABLE_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DGTSAM_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DMEX_COMMAND:STRING=/usr/local/MATLAB/R2020b/bin/mex
CXX_FLAGS=-DEIGEN_DONT_VECTORIZE -DEIGEN_DISABLE_UNALIGNED_ARRAY_ASSERT
C_FLAGS=

UNSTABLE ON, MATLAB UNSTABLE ON, PYTHON ON

Builds / completes / unit tests / matlab tests [ Focal x86_64 / Matlab2020b1 / Python 3.8.5 ]

varunagrawal · 2021-03-26T06:09:21Z

Are you using a virtual environment?

varunagrawal · 2021-03-26T06:10:21Z

Basically what is happening is that CMake is unable to find the python package on your system. It needs that to specify the interpreter and python versions

carlmorgan · 2021-03-26T06:59:04Z

Basically what is happening is that CMake is unable to find the python package on your system. It needs that to specify the interpreter and python versions

No venv involved. Stock ubuntu python tooling. Looks like CMake is finding it.

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version        Architecture Description
+++-==============-==============-============-================================>
ii  python3        3.8.2-0ubuntu2 amd64        interactive high-level object-or>

which python
/usr/bin/python

ls -l /usr/bin/python
lrwxrwxrwx 1 root root 7 Apr 15  2020 /usr/bin/python -> python3

python --version
Python 3.8.5```

### cmake
-- Checking Python Version
-- Setting Python version for wrapper
-- pybind11 v2.6.0 dev1

--  Build GTSAM unstable Python                      : Enabled

-- Python toolbox flags
--  Build Python module with pybind                  : Enabled
--  Python version                                   : 3.8

CMakeCache.txt

PYTHON_EXECUTABLE:PATH=/usr/bin/python3.8

//Path to a file.
PYTHON_INCLUDE_DIR:PATH=/usr/include/python3.8

//Path to a library.
PYTHON_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.8.so

//Path to a library.
PYTHON_LIBRARY_DEBUG:FILEPATH=PYTHON_LIBRARY_DEBUG-NOTFOUND

//Path to a library.
Python3_LIBRARY_DEBUG:FILEPATH=

//Path to a library.
Python3_LIBRARY_RELEASE:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.8.so

//Path to a library.
Python_LIBRARY_DEBUG:FILEPATH=

//Path to a library.
Python_LIBRARY_RELEASE:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.8.so

It is also building the matlab_wrap.py without issue. I'll try digging further and let you know. Understood it shouldn't be needed.

carlmorgan · 2021-03-26T09:44:21Z

Testing on 92ce070

OS	UNSTABLE	Unit Tests	MATLAB	UNSTABLE MATLAB	MATLAB Unit Tests	PYTHON
Xenial	GOOD	PASS	Not Tested	Not Tested	Not Tested	Python 2.7.12 FAIL
Bionic	GOOD	PASS	Not Tested	Not Tested	Not Tested	Python 2.7.17 FAIL
Focal	GOOD	PASS	2020b1 GOOD	2020b1 GOOD	GOOD	Python 3.8.5 GOOD

CMAKE_ARGS=-DGTSAM_USE_SYSTEM_EIGEN:BOOLEAN=FALSE -DGTSAM_WITH_EIGEN_MKL:BOOLEAN=FALSE -DGTSAM_WITH_TBB:BOOLEAN=FALSE -DGTSAM_BUILD_EXAMPLES_ALWAYS:BOOL=FALSE -DGTSAM_BUILD_UNSTABLE:OPTION=ON -DGTSAM_USE_QUATERNIONS:BOOL=TRUE -DGTSAM_BUILD_TESTS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DGDB:BOOL=TRUE -DDEBUG:BOOLEAN=TRUE -DASSERT:BOOLEAN=TRUE -DASSERT2:BOOLEAN=TRUE -DGTSAM_BUILD_PYTHON:BOOL=TRUE -DGTSAM_UNSTABLE_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DGTSAM_INSTALL_MATLAB_TOOLBOX:BOOLEAN=true -DMEX_COMMAND:STRING=/usr/local/MATLAB/R2020b/bin/mex
CXX_FLAGS=-DEIGEN_DONT_VECTORIZE -DEIGEN_DISABLE_UNALIGNED_ARRAY_ASSERT

Haven't been able to run the python unit tests yet - can't get it to install to non system directories etc.

.py chmod / CMake python executable issue not seen again on latest PR so I'll put down to 'ghosts'

varunagrawal · 2021-03-26T10:29:12Z

I wouldn't count on python 2 working for you. It's EOL and unsupported so best to upgrade or ignore it.

Also, something to note is that the wrap directory is a git subtree so changes have to be first made and approved in the wrap repo. There might be some disconnect between when I pulled in some fixes and you tested the wrapper.

At this point I'd like to land this PR and then take your feedback forward in the next one.

ProfFan · 2021-03-26T13:31:58Z

Xenial and bionic both have python3, so could you test against the system Python 3 instead?

dellaert

This PR is hard to review: Is it simply pulling in the new wrap from the other repo? In that case, I'll approve modulo CI succeeding. If there are any other changes (to .i files etc) I think they should go in a subsequent PR.

varunagrawal · 2021-03-26T16:38:09Z

No changes to the .i files, but some CMake changes to get it to work correctly and address the issues @carlmorgan brought up.

Also, 92ce070 has all the latest fixes for the chmod/permission-denied issue so super glad to see it works for you.

carlmorgan · 2021-03-26T21:13:15Z

Thanks for the merge - great to have everything back in a working state.

@varunagrawal @ProfFan
WRT various combinations of systems python / OSs versions etc. We don't currently need / use UNSTABLE or Python but I try to keep a 'upstream develop' CI branch running with as much enabled as works - I was just trying to provide a little more testing feedback on what I knew did work...

CMake will find 'python' but if the requirement for a support version is version 3.5+ (/whatever) it would be good to report at configure time (rather than compilation) - not an 'issue' per-se - I'm just being a 'user' :)

From what I can see in the docs - official supported platforms are 16.04 & 18.04 which now are getting older - is there a plan for making 20.04 a primary platform?

varunagrawal · 2021-03-26T21:16:11Z

@carlmorgan you're correct and I agree with you. It should be reported at configuration time and we should make that happen soon.

Thank you so so much for all your feedback though. It's been invaluable in fixing all the small bugs that crept in wrt the new wrapper. 🙂

varunagrawal · 2021-03-26T21:17:04Z

I run GTSAM on Ubuntu 20.04 so it definitely works on there. Documentation is another thing that needs improvement, but for now let's enjoy a well-deserved weekend.

varunagrawal added 3 commits March 24, 2021 00:35

more specific version of python to test against

8db1eed

Merging 'master' into 'wrap'

54e5dc2

varunagrawal added python Related to python wrapper matlab Related to MATLAB wrapper labels Mar 24, 2021

varunagrawal requested review from ProfFan and dellaert March 24, 2021 04:57

varunagrawal self-assigned this Mar 24, 2021

varunagrawal added 5 commits March 24, 2021 16:14

Merging 'master' into 'wrap'

37b5b66

added CMake variable to set the matlab.h include

30c84ea

Merging 'master' into 'wrap'

04b9625

switch back to simple python3

82d6e68

fix number of arguments to matlab_wrap for unstable

b7e19d6

varunagrawal force-pushed the fix/wrapper-update branch from 399c30c to 729953e Compare March 25, 2021 17:30

varunagrawal added 2 commits March 25, 2021 13:32

Merging 'master' into 'wrap'

1e8973e

varunagrawal force-pushed the fix/wrapper-update branch from 729953e to 1e8973e Compare March 25, 2021 17:32

varunagrawal added 2 commits March 26, 2021 00:54

Merging 'master' into 'wrap'

92ce070

dellaert reviewed Mar 26, 2021

View reviewed changes

dellaert approved these changes Mar 26, 2021

View reviewed changes

dellaert merged commit 995c049 into develop Mar 26, 2021

dellaert deleted the fix/wrapper-update branch March 26, 2021 19:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrapper Update #727

Wrapper Update #727

varunagrawal commented Mar 24, 2021

varunagrawal commented Mar 24, 2021

carlmorgan commented Mar 24, 2021 •

edited

Loading

dellaert commented Mar 24, 2021

varunagrawal commented Mar 25, 2021

carlmorgan commented Mar 25, 2021

carlmorgan commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021 •

edited

Loading

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

carlmorgan commented Mar 26, 2021 •

edited

Loading

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

ProfFan commented Mar 26, 2021

dellaert left a comment •

edited

Loading

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

Wrapper Update #727

Wrapper Update #727

Conversation

varunagrawal commented Mar 24, 2021

varunagrawal commented Mar 24, 2021

carlmorgan commented Mar 24, 2021 • edited Loading

dellaert commented Mar 24, 2021

varunagrawal commented Mar 25, 2021

carlmorgan commented Mar 25, 2021

carlmorgan commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021 • edited Loading

varunagrawal commented Mar 25, 2021

varunagrawal commented Mar 25, 2021

carlmorgan commented Mar 26, 2021 • edited Loading

unstable matlab ON

Python ON, Unstable Matlab OFF

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

UNSTABLE ON, MATLAB UNSTABLE ON, PYTHON ON

varunagrawal commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

CMakeCache.txt

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

ProfFan commented Mar 26, 2021

dellaert left a comment • edited Loading

Choose a reason for hiding this comment

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

varunagrawal commented Mar 26, 2021

carlmorgan commented Mar 24, 2021 •

edited

Loading

varunagrawal commented Mar 25, 2021 •

edited

Loading

carlmorgan commented Mar 26, 2021 •

edited

Loading

dellaert left a comment •

edited

Loading